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CHAPTER I 
INTRODUCTION 




Developnent of this pilot training project was stimulated by 
observation of the growing demand In social work for persons trained to 
do research and the relatively limited special efforts being made to 
meet this need. The writer knows of no formal study of the relative 
number of positions for whom social work researchers are being sought, 
nor the relative number of social work researchers looking for Jobs, 
There appears to be general agreement, however, that there Is a serious 

shortage , 

That social workers enter the field largely because they are 
interested In working with people and want to provide service to them, 
rather than because of their Interest In developing knowledge. Is an 
assumption which few social workers will dispute. At least one formal 
Study supports this sssutnptlon* 

The official recruitment efforts of the profession also stress 
the opportunities for giving service rather than the opportunities for 
developing knowledge,^ This emphasis on the helping possibilities In 



’^Amulf Pins, ’’Who Chooses Social Work, When and Why, p, 87, 
shows that 62 per cent of the persons who entered social work In I960 
entered because they considered they would be making a contribution to 
an Individual or society, or because they liked to work with peopl®* 
None gave answers that showed they wanted to learn how to contribute to 
developing knowledge, but this may have been due to the form of the 
questions. 



^See "Summer of Decision,*? by the Council of Social Work Educa 
tlon, a 35 mm film, 1959, 
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the field Is certainly valid for a profession whose chief reason for 
being Is to provide assistance for people In distress, providing It does 
not have as a corollary any nonetnphasls on the development of knowledge 
on which competent professional performance Is based. 

The pilot training plan described In this report assumed that the 
focus of recruitment efforts of the profession would remain the same In 
the near future; that Is, it expected that recruitment would continue to 
emphasize the service-providing rather than the knowledge-building func- 
tions of social work. This project further assumed that social work 
knowledge should be developed by social workers; and that, at least In 
the near future, persons with Master of Social Work degrees could and 
must take a major role In building and extending this knowledge through 
research. This latter consideration. In particular, was central to the 
broad goals of the project; Identifying and maximizing research learn- 
ing potential of students In social work Master*s degree programs. 

Specifically, this project sought to learn how, at the time of 
admission, schools of social work could begin to Identify various levels 
of potential, flair for, and Interest In doing research among students. 

At the same time. It tried to learn what were some of the teaching 
methods and course content most likely to maximize potential at various 
levels. In the process. It expected to test and refine an Instrument 
that could be used both to Identify levels of potential and to measure 
learning achieved. 

In designing this training project, the writer and others Involved 
were aware that many teachers do not view the research classroom course 
or courses and the experience In research provided students (often called 
the research project or practlcum), as separate entitles. For several 
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reasons, hotiever, a decision was made to concentrate exclusively on the 
classroom courses in research at the five schools concerned. 

First, success in any research experience such as a project or 
thesis was considered to be dependent on success in the course. Evi« 
dence for this is the almost universal requirement among schools of 
social work that students take at least one basic classroom course in 
research prior to any research experience. Second, the classroom course 
lent itself more readily to objective evaluations of the outcome of 
training than the project. Examinations using multiple choice questions 
vere more likely to be used and acceptable to students as a basis for 
course grades, than in the project or thesis, where any grading was 
based on the narrative report itself. Third, findings from this pilot 
training plan for the research classroom course were expected to be 
useful in developing subsequent training plans for research projects 
and theses. 

This pilot training project must be considered exploratory. 

It concentrated on study of methods of measuring potential for learning 
research among social work students and on study of the influences on 
student learning from existing differences in research teaching rather 
than from planned experimentation. Use of existing statements of 
content and objectives as data sources for differences in teaching 
resulted in some lack of detail on these topics. Consequently, findings 
are more general than specific. Nevertheless, it served its purpose 
by indicating that differences in potential could be identified, and 
by offering suggestions for changes in research teaching that could 
maximize this potential for each student. 

Data on 353 students at the five participating schools were 
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obtained during 1965-66 prior to thaae atudenta having any claaaroom 

training In research. Data on 263 students v«re obtained after they had 

3 

completed their classroom work In research In 1966 and 1967. Student 
attrition and some difficulty In obtaining some of the material at two 
of the schools %iere responsible for the losSy but the 90 students lost 
did not appear to effect the representativeness of the sanq>le. Analyses 
of background data and test results showed that these 90 students v«re 
not significantly different at the 5 per cent level from those who com- 
pleted the study. 

The five schools, selected for their variety In size, geographical 
location, source of support, and length of accreditation, provided ll 
teachers who taught a total of 14 sections. 

The training plan exploited a number of naturally occurring dif- 
ferences among the five schools that Influenced student learning. These 
Included such factors as the relative emphasis on various aspects of 
content, differences In methods of teaching, and In learning experi- 
ences, variations In methods of evaluating learning achieved, the time 
and pl^ce of research In the curriculum, and tfjacher characteristics. 

As part of this training plan, two schools set up experimental sections 
where new content was Introduced or new methods of teaching attempted, 
or both. Little or no change was made In the teaching at the other 
three schools. 

Data on the objectives, methods of teaching research, course 



^ata at the various schools were collected at different times 
because the research courses did not all begin and end In the same 
semester. At each school data were collected prior to any classroom 
work and subsequent to Its completion. 
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content and other training details were obtained through interviews 
with a representative from each school and from the course outlines and 
statements of objectives, etc., prepared by each teacher. 

Student character istica prior to taking any research course work 
were obtained by questionnaires and a test. Two questionnaires obtained 
biographical data about students. A measure of students* attitude to 
and knowledge of research was obtained by means of a test that the 
writer had been developing during the last five years. This instrument, 
called the "Measurement of Attitudes and Research Knowledge," and 
referred to as the MARK, (by the first initial of each letter of the 
title) had been found in several studies at Tulane to predict which 
students did best in the research course, and which could benefit from 

A 

special instruction. 

Further material on this instrument will be available in a 
companion monograph on this training plan, to be published approxi- 
mately July, 1968. 

Subsequent to students completing the beginning research course, 
but prior to any work on the thesis, research practicum or project, the 
instrument measuring attitudes to and knowledge of research (the MARK) 
was administered again. In addition, students were given an examination 
containing certain specialized and somewhat more difficult than average 
questions about research to determine tdiat specialized learning had 
taken place. To get further information on teaching methods, at the 
same time, students completed a questionnaire evaluating the teaching 



Slarris K. Goldstein, "Student Social Work Characteristics Re- 
lated to Potential for Learning Research," 1965, mimeod material. Also 
see other papers on this instrument by various research apprentices of 
the writer produced during 1962-63-64, and available in mimeod form 
from the writer. 
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at each school and section and noted some of their general reactions to 
research. Final grades In the research course completed the evaluative 

data. 

Analysis was guided by the objectives of the study. Using data 
obtained from the Instruments administered after the course as crlterls, 
determinations were made of the ability of the various pre-course Instru- 
ments and questionnaires to Identify potential for learning research. 
Changes In MARK scores, special test questions, and findings In the course 
evaluations, as measures of learning achieved, were then analyzed for 
their relation to the teaching variables. 

In the material that follows. Chapter II presents some historical 
material about the profession's concerns with the teaching of research 
and some problems In and theory for Identifying and predicting success In 
learning research. Chapter III explains how the pilot training plan was 
carried out and how the data used to evaluate Its success or failure were 
obtained. Chapter IV provides some data on students prior to training. 

In Chapter V. Information on the teaching variables that were part of the 
overall training pirn is presented. Chapter VI discusses the adequacy of 
the Instruments used to evaluate learning. Chapter VII gives the out- 
come of the training at the participating schools, together with an 
evaluation of results. Chapter VIII discusses further training and 
evaluation suggested by this project. 

To preserve anonymity, all participating schools are designated 
only by number. The size of the sample at each school, the location, 
and other Information that might Identify schools has been omitted. 
Detailed tables on each school (Identified by number only) on all vari- 
ables analyzed In this study may be obtained from the writer by 



Interested researchers 
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CHAPTER II 



STUDIES OF RESEARCH TEACHING AND SCIENTIFIC PRODUCTIVITY, 
AND THEIR RELATION TO STUDY METHODOLOGY 



This chapter discusses some of the past and current concerns 
relating to the teaching of social work research. It presents findings 
of other studies aimed at Identifying scientific productivity and 
creativity, and Indicates why the particular methodology used In this 
study was selected. 

The practical goals of this training plan differed in focus from 
some of the profession's previous concerns about the teaching of 
research. They did, however, have some similarity to attempts In other 
fields to learn what makes for success In research and to some current 



social work explorations of how to maximize learning for different 

1 



levels of student ability. 



Previous Concerns With Research Teaching 
At first glance, the concerns expressed most commonly In the 
literature about the teaching of social work research are not the ones 
on which this training plan was focused. Further assessment of these 
concerns, however, does show the relevance of this project to the 
profession's needs. 



^At the Annual Program Meeting of the Council on Social Work 
Education In 1966 there were workshops on teaching the gifted and 
disadvantaged student. 
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The two topics most frequently mentioned in articles about the 
teaching of research are integration of the content taught in research 
with the remainder of the social work curriculum and methods for teaching 
the research project or practicum that will maximize student interest. 

At times, the former question has been expressed as a concern how to pro- 
vide an educational plan in which an attitude of scientific inquiry and 
ability to use the scientific method in problem solving will permeate the 

entire curriculum. How to give students an appropriate research experi- 

2 

ence has been the focus of the second interest. 

The writer and others involved in this training plan recognized 

the importance of these previous topics. It seems clear, however, that | 

I 

[ 

integration of the research sequence and the remainder of the curriculum > 

is related to how the research course is taught and how the rest of the 
curriculum is taught. The thesis of this training plan was that if 
better research teaching takes place, problems in integrating research 
with the rest of the curriculum and in developing and maintaining 



^The most sustained concern relative to teaching research has 
been manifest in documents of the Council on Social Work Education. 
Review of CSWE workshop reports from 195A on "Social Work Research," 
from 1957 on "Social Welfare Research and Demonstrations," and from 
1964 on "New Approaches to Administration and Research in Social Work 
Education," all support the conclusion that integration of research 
and the teaching of the project have received major attention. Only 
one out of seven papers in "Selected Papers and Methods of Teaching 
Research in the Social Work Curriculum," published by the Council on 
Social Work Education in 1963 discusses the teaching of the research 
course. Four of these papers were about teaching the project, one 
about the post-master's program, and one about integrating the teach- 
ing of research and other practice methods. 
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Interest in the practlcum will be reduced. 

No material could be located In the literature related to the 
writer's concern for Identifying and maximizing research learning 
potential. Correspondence with other faculty did locate two teachers 
with similar Interests. Professor Irving Lukoff at the University of 
Pittsburgh, and Professor Merlin Taber at the University of Illinois 
both reported that they had been Interested In the variations In 
knowledge that students brought to their research courses. Dr. Lukoff 
has done some beginning work on a test similar to the one developed by 
the writer to Identify these differences. Dr. Taber modified Dr. 

Lukoff *s test and did some further Investigation with It. As far as 
the writer can determine, however, development of these two Instruments 
Is still In progress, and no reports describing even preliminary find- 
ings are available. 

Dr. Lawrence Northwood attempted to enrich the training of 
certain students In the Master's program but his design differed from 
the training plan of this project In three ways: In the way students 

were selected for more Intensive training. In the method used to pro- 
vide this training, and In how the outcome was evaluated.^ This study, 
carried on at the University of Seattle, selected students for 



i 






3 

Some current writings Imply this though they do not state It 
explicitly. See R.M. Titmus, "The Relationship Between Schools of 
Social Work, Social Research and Social Policy," Journal of Education 
for Social Work . Spring, 1965, pp. 68-75, and E. Walsh, "Research and 
Teaching Casework," Journal of Education for Social Work. Fall. 1965. 
pp. 47-52. 

^Lawrence K. Northwood, "Enriched Training In Research In the 
Master's Degree Program," Social Work Education Report. June, 1966. 
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additional training In research on the basis of variables which appeared 
to have face validity* Including their stated Interests, completion of 
one or more courses in behavioral science research, employment In research 
positions, and undergraduate grade point averages. Methods used to maxi- 
mize the potential of these students Included providing them with a class- 
room course In addition to the basic research course, and considerable 
additional work In a special practlcum during the summer. Evaluation of 
students' performance was made on the basis of a paper, classroom partici- 
pation, and a final examination (kind unspecified). 

The relatively few reports of previous attempts to work toward 
objectives similar to this training plan, however, does not mean that 
there Is agreement on how to Identify differences In students' abilities 
nor even that there Is any consensus on what the content, emphasis, or 
teaching methods In social work research should be. 

The Research Workshops of the Council on Social Work Education 
previously cited, raise such questions as "How much statistics should be 
required, and how much statistics should be taught as part of the research 
course? Should the research course be taught In the first or second 
semester or In the first or second year? Should there be a full professor 
whose time Is allocated to the teaching of research or should all faculty 
participate In this function?" 

That definite answers are yet to be found is evidenced from a 
study of school catalogs. The wide variety of content mentioned, hours 
of credit given, the semester In which research Is taught, and the 
experience and background of persons teaching th^ course among the more than 
60 schools of social work Is an Indirect acknowledgement that the profession 
Is far from having decided what is the best plan of teaching even a single 
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course, much less of Integrating the research sequence with the rest of 
5 



the curriculum. 



Theoretical and Empirical Bases for Predicting 
Success In Research 



While there has been little study In social work of the factors 
within students making for success In learning research, or little In- 
vestigation of how to teach research In a way that makes It most likely 
these factors can operate, there has been considerable investigation 
In other fields of factors Influencing success In scientific effort. 
These efforts have generally been aimed at determining which factors re. 
late to scientific creativity or which factors can predict research 
productivity subsequent to the completion of graduate work. 

An excellent summary of theoretical formulations and empirical 
findings related to these topics is found In a report on the Utah Con- 



ferences on Creativity where workers from different disciplines In 

6 



various parts of the country brought their knowledge together. 

While the objective of this training plan was not to Identify 
or maximize potential for creativity, per se, among social work 
students, studies of the factors related to creativity were considered 
relevant to Its goal. This relevance was based on the Idea that 
factors Influencing research creativity and productivity are similar to 
those that Influence learning substantive content about research. 



^Julanne Haspel, unpublished material, "Variations In Master s 
Research Teaching at Schools of Social Work," available In mimeographed 
form from the writer. 



^Calvin W. Taylor and Frank Barron, Scientific Creativity. It ^ 
Recognition and Development . New York: John Wiley and ons. 
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Theoretical formulations and empirical studies aimed at determining 
the factors Influencing scientific creativity and productivity were found 
to deal with three kinds of variables related to Intellect, personality, 
and culture, respectively. 

Studies of productive scientists have been carried out by various 
methods ranging from clinical Interviews aiid projective techniques, through 
empirically developed biographical Inventories and factor-based tests. 
Various criteria of scientific creativity and productivity have been In- 
cluded, ranging from the numbers and kinds of research publications to 
supervisory ratings of personnel. From all of these various studies, a 
consistent picture of the personality and Intellectual characteristics of a 
successful scientist emerges. The successful scientist possesses: 

"1. A high degree of autonomy, self-sufficiency and self-direction. 

2. A preference for mental manipulation Involving things rather 
than people. Accompanying this is a distant or detached attitude In 
Interpersonal relationships and preference for Intellectually challenging 
situations rather than socially challenging ones. 

3. High ego strength and emotional stability. 

4. A liking for method, precision, and exactness. 

5. A preference for such defense mechanisms as repression and 
Isolation In dealing with affect and Instinctual energies. 

6. A high degree of personal dominance but a dislike of per- 
sonally toned controversy. 

7. A high degree of control of Impulse, amounting almost to 
over-control; relatively little talkativeness, gregariousness. 
Impulsiveness. 

8. A liking for abstract thinking with considerable tolerance of 



cognitive ambiguity 
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9. Marked independence of Judginent» rejection of group pressures 
toward conformity and thinking. 

10. Superior general intelligence. 

11. An ejirlyy very broad interest in intellectual activities. 

12. A drive toward comprehensiveness and elegance in explanation. 

13. A special interest in the kind of 'wagering* which involves 
pitting oneself against uncertain circumstances in which one's own 
effort can be the deciding factor."^ 

This cluster of traits may require some modification to describe 
a successful social work scientist^ possibly modification toward more 
interest in people and a change toward more interest in social relation- 
ships. Otherwise, it appears to have utility for identifying and pre- 
dicting those persons who will likely have most potential for learning 
social work research. 

A small number of sociological and cultural variables have been 
found to be associated with scientific productivity. 

Students from rural areas and small towns, those whose families 
are in other than white collar occupations, and those with lower- 
middle class status tend to be most productive. Colleges and universi- 
ties in the mid-west and far west, of moderate but not necessarily 
lowest cost, appear to produce most productive scientists. Student 
cultures characterized by humanism, breadth of interest, and reflective- 
ness were found to be associated with scholarly productivity, and 
cultures characterized by participation and aggression were negatively 
related to this productivity. Natural science productivity, on the 

^Taylor and Barron, op.cit . , p. 385. 
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other hand, was found associated with student cultures stressing 

1 

\ scientism and aggression. Outstanding achievement in natural sciences 

( 

I docs not appear in colleges where student cultures stress social con- 

j formity*® 



Teachers who are most successful in encouraging students to learn 
research and to become productive scientists are those who are said to 
have such traits as demanding high standards of performance, assertive- 
ness, and entrepreneurial vigor. Breadth of interest, administrative 

Q 

participation* and teaching zest, are additional assets.' 

Identifying Research Potential in This Training Plan 
In this training plan, the problem of identifying levels of potential 
for learning research might have been approached by examining the large 
number of intellectual factors already identified, and for which psycho- 
logical tests have been developed. An alternate method would have been 
to use existing personality tests to learn the extent to which social 
work students possess each of the personality factors identified by 
various investigators. 

Two problems were considered to preclude the use of either of 
these approaches. Either method would have required some criterion of 
success in learning social work research. No satisfactory criterion of 
known reliability was available; classroom grades were not considered 
suitable for this purpose. In addition, the writer has not found 

®Donald L. Thistlewaite, "The College Environment as a Determi- 
nant of Research Potentiality,” in Taylor and Barron, pp. 265-277. 

Robert H. Knapp, "Demographic, Cultural, and Personality 
Attributes of Scientists," in Taylor and Barron, p. 210. 
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standard psychological tests fruitful in making predictions about social 
work students, The high degree of homogeneity among social %«rk 
students, in terms of their ability and personality, reduces the predic- 
tive power of tests which predict satisfactorily in more heterogenous 
populations where the range of test scores is broader. 

On the basis of the empirical studies referred to, the writer 
thus developed a simple questionnaire to assess biographical data in 
the familial, cultural, and educational area. This was supplemented by 
a measure of interests, on the grounds that personality variables as 
well as some background influences would be manifest indirectly through 
expressed interests. 

The decision to place major emphasis on using the MARK in this 
study to identify potential for learning research was based on the 
following line of reasoning: 

The writer assumed that in the course of social work students' 
life experiences prior to attending a graduate school of social work, 
the amount of knowledge they obtained about research and the attitudes 
they developed to it would be influenced by both their personality 
make-up and their intellectual abilities. Students with the intellec- 
tual ability and/or personality traits likely to facilitate success in 
learning research were considered likely to have learned more research 
over their life experiences and to possess more positive attitudes to 
it, than students who did not have these intellectual abilities or 
personality traits. While opportunity to learn research and to develop 

^^arris K. Goldstein,"Prediction of Success in Schools of Social 
Work,”SumnBary of Research, George Warren Brown School of Social Work, 
Washington University, 1959-60, 
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positive attitudes to it could be considered an intervening variable in 

the equation above, the writer further believed that students with the 

intellectual ability and personality traits associated with success in 

learning research would, in the course of their life experiences, tend 

to f ind such opportunities. This latter assumption is supported by 

theories of vocational choice and by the experience of the writer. He 

has found that some students could not understand why they had scored so 

high on the MARK before taking the research course, since they had no 

background in research. Interviews with these students invariably 

brought out the fact that they had voluntarily sought out the kind of 

material which would tend to increase their knowledge of research and | 

i‘ 

[ 

positive attitudes to it. For example, these students tended to read 
such literature as science journals, science feature articles in the 
newspaper and science fiction. 

A test like the MARK that measured knowledge and attitudes was 
thus considered an indirect method of identifying those intellectual 
abilities and personality traits likely to influence potential for 
learning research. At the same time, the MARK provided an economical 
method of assessment of the outcome of this training plan, because it 
could serve as a criterion for what students learned in the research 
course. The MARK thus served a double purpose and provided: (1) a 

uniform method of measuring attitudes to research and what students j 

had learned about research prior to taking the research course at all 
five schools; (2) whether any changes in their attitudes to and 
knowledge about research had taken place after they were taught social 

r 

work research. i 
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CHAPTER III 



methodology of the training plan 



This chapter gives a summary of methods used to get data on 
students before and after they were taught research and to obtain 
Information on teaching objectives, course content and methods used 
to Implement student learning. 



Method of Obtaining Data on Students Prior to 
Their Taking Research Course 



Data obtained on students prior to their enrollment In the 
research course were obtained for two purposes that were In line with 
the overall objectives of the training plan. First, the aim was to 
obtain data considered likely, on the basis of material discussed In 
Chapter II, to predict different levels of potential for learning 
research. Second, some of the data was to be used as a base line from 

which to measure learning achieved. 

Because the writer had been attempting to evaluate his own 
teaching since 1961, three previously tested Instruments for gathering 
data were available for use In this pilot study. These Instruments, 
details on which are given below, were as follows: 

1. The first was a questionnaire which obtained Identifying 
Information on students* undergraduate college experi- 
ence as well as material about their family and cultural 
background. In the remaining chapters of this report, 
this questionnaire Is referred to as the "I.D. 
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Questionnaire” and the information obtained from it is called 
*'I.D. data.” 

2. The second was a questionnaire which obtained certain informa- 
tion on students' interests, spare time activities and hobbies. 
This questionnaire is referred to as the "Interest Test" and 
information obtained from it is called "Interest data." 

3. The MARK, a test measuring attitude to and knowledge of 
research was the third instrument. Data obtained from it 
includes an attitude score, a knowledge score, and a total 
score obtained by combining these other two scores. 

The I.D. Questionnaire: 

The I.D. Questionnaire was developed by the writer in 1963 to obtain 
data on students* educational and cultural background. 

Items initially selected for this instrument were those that 
appeared to have face validity; others were suggested by the studies of 
success in scientific endeavor in other fields such as psychology, 
medicine, and physical science, referred to in Chapter II. Items of the 
face validity type were those that asked whether students had course work 
in statistics or research while an undergraduate, or those asking for a 
self-assessment on attitudes to and ability in mathematics and abstract 
reasoning. To these were added questions on demographic variables, such 
as size of hometown and geographical area of students' origin and cul- 
tural variables such as social class. 

On the basis of item analyses carried out in 196A and 1963, certain 
of these items which did not correlate with final research course grades 
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at Tulane were eliminated. Other items were modified and some new 
items added 

In the form used in this pilot training plan, this I.D. 
Questionnaire consisted of 32 pre-coded items. It was administered 
to students as a group at each of the five schools before they had 
any course work in research at the school of social work. In general, 
students completed this questionnaire in about 20 minutes. 

The Interest Test: 

This test was originated by the writer in 1963, at the same 
time as the I.D. Questionnaire. Because so little was known regard- 
ing vhat interests might relate to different potentials for learn- 
ing research, it purposefully tried to cover a broad range of 
interests. Use of a broad rather than a specific test was supported 
by what is known of other interest tests, where in some cases, 
rather unexpected correlations were found between certain interests 
and success in a field. The test format asked the student to check 
those activities he had "spent some time on, either as a paid worker 
or as a voluntary or recreational activity in the last five years." 

A list of activities was provided. 

Originally, this test contained 60 items. In 1964 and 1965 
other items were added similar in content to those original ones 
that correlated with research course grades. At the same time items 

^Mimeographed papers by Douglas Radabaugh, Julanne Haspel, 

Faye Weckel, and Eugene Hadley, Student Research Apprentices, who 
worked with the writer on these analyses are available to Interested 
persons. Only those items that correlated at the 5 per cent level 
with research course grades or those supported by other theoretical 
or empirical studies were retained on the questionnaire. 
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showing less predictive value were eliminated. 

This test was administered immediately following the I.D. Ques- 
tionnaire to students at each of the five schools as a group. In the 
form used, it consisted of 6A interest items to be answered "yes" or "no" 
and required about ten minutes for students to complete. 

The "Before" MARK: 

The writer began work on this test in 1961 at Washington University. 
Work on this test was stimulated by the belief that students brought to 
the research course a far wider range of knowledge and attitudes than they 
brought to other courses, a range that posed special problems in teaching 
research. By administering the MARK before the research course, the 
writer attempted to identify various levels of student potential for 
learning research as measured by course grades on objective tests. By 
administering it again after the course, test scores served as an indi- 
cator of the amount of learning achieved. 

In 1961, the original MARK consisted of approximately 30 items 
taken from multiple choice final examination questions of the writer. 
Students' pre-course scores on this original test confirmed the writer's 
belief about the relatively wide range in knowledge that students 

3 

brought to the course. Some students scored sufficiently high so that 
had this been a bona fide final examination, they would have been given 
a grade of "B+," Others scored so low that their score was less than 

2 

See papers previously referred to by student apprentices# 

A Study by the writer of variations in casework knowledge prior 
to students taking any casework courses showed considerably less varia- 
tion in the casework area than in research. See "The Tulane Assessment 
Scale for Caseworkers," Tulane Studies , Vol. VII, by the writer. 
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15 per cent of the score made by the students. (On examinations 

at the end of the course, even the failing students generally score at 

least 50 per cent as much as students.) 

In succeeding years at Tulane, this original test was modified 
and enlarged on the basis of item analysis, or correlations of each 
item with every other item, and with grades on objective testa at 
various points in the research course. In 1964, certain items con- 
sidered to measure attitudes as well as knowledge were added. These 
original ''atcitude*' items were selected on the basis of the writer's 
inferences which attitudes appeared to make a difference in student 
learning. In 1964 and 1965, the predictive value of these items was 
also tested by item analysis. 

In the form used in this pilot training project, the MARK had a 
total of 59 pre-coded, multiple choice questions, of which 36 were 
Considered to measure "attitudes" and 23 to measure "knowledge." Atti- 
tude and knowledge items were distributed at random throughout the 
first 40 items of the test. The last 19 items were "knowledge" items. 
The MARK was administered to students at each school in a group before 
they had taken any research courses. 

In the material that follows, this form of the MARK is referred 
to as the "before" MARK to differentiate it from that slightly differ- 
ent form administered after students finished their research course. 

Method of Obtaining Data on Students Subsequent to Their 
Completing the Research Course 

At the end of this pilot training plan, four kinds of information 
were obtained. Three of these also capitalized on previously developed 
instrumentation of the writer. 
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1. A slight modification of the MARK called the "after" MARK pro- 
vided knowledge about and attitudes to research. 

2. A group of questions found by experience to be unusually dif- 
ficult for Master's level students were used to obtain information on any 
learning above and teyond that usually expected at the MSW level. These 
questions^ objective in form» are referred to as "Special Test Questions^" 
or the STQ. 

3. A questionnaire asked for students' reactions to the teaching 
and content of the course. This is referred to as the "Course Evaluation." 

4. Grades in research courses, based on whatever method each 
teacher used in assigning grades, were obtained. 



The "After" MARK; 

With slight modifications, the MARK was used to obtain an attitude 
score, a knowledge score, and a total score that would show changes in 
these three variables after the students completed their research course 
work. The "after" MARK differed from the "before" MARK in only four 
items. In the "before" MARK, these items had dealt with student expecta- 
tions about the research course; in the "after" MARK these items were changed 
so that they asked about student experiences rather than expectations. 

For example, before items asked about apprehension about research, or 
about the expected application of the students' learning to their prac- 
tice. Corresponding items in the after form asked if the course had 
caused as much anxiety as expected, or if application of learning was 
more or less than expected. 

The "after" MARK was given the students at Schools 1,2,4, and 5 
in a group administration immediately after students finished their 
research course. At school 3, students were asked to take the MARK home 
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with them for the sunraer and return it by mail. Some attrition occurred 
at each school, so that after data was not obtained on all students on 
whom before data was available. For details, see Chapter VII. 

The Special Test Questionnaire: 

In 19o5 the writer developed some questions aimed at learning 
whether students who had been placed in a "special" section and exposed 
to certain additional concepts (see Chapter V for details) had learned 
any additional content. Analysis of student responses to these ques- 
tions at Tulane showed them to be of more than average difficulty. No 
student received a score of more than 80 per cent of the items correct, 
and the average score was about 50 per cent correct. These "special" 
questions were used in this pilot training project to evaluate if any 
learning, above and beyond that generally expected, had taken place. 

In the form used to evaluate the success of this project, there 
were 20 of these questions, all multiple choice. They cut across 
research methodology by presenting 20 possible procedures to be used 
in a predictive research study. Four possible outcomes of each proce- 
dure were provided. These were (1) whether following the procedure 
would produce an increase in the reliability with which the predictor 
variable could be observed, but have no influence on the validity of 
predicting the expected relationship; (2) whether the procedure would 
increase the accuracy of prediction of the criterion variable, but 
have no influence on the reliability with which the predictor variable 
was observed; (3) whether the procedure neither increased the 
reliability or validity (including those whose influence was unknown 
or which were likely to decrease reliability and validity) of the pre- 
diction; and (A) whether the procedure would both increase the 
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reliability with which observations of the predictor variable could be 
siade and at the same time Increase the validity with which the prediction 
could be determined. Students were asked to select which outcome of each 
procedure was most likely. 

At school 5, these questions were administered as a part of the 
final examination. At schools 1»2, and 3, they were given In the Fall of 
1966. They were not administered at school 4. 

Course Evaluation: 

Since 1957, the writer had been using a questionnaire administered 
to students at the last class session of the research course, to obtain 
students' reaction to teaching methods In general, to specific aspects of 
content, and to get students' general reaction about research and the 
learning they had achieved. 

Like the other Instruments In this study, this questionnaire was 
modified from year to year. Modifications were made, however, on the 
basis of logical rather than statistical analysis. The Instrument was 
updated In keeping with changes In the content and teaching methods of 
the course taught by the writer. It Is believed these changes made It 
more Inclusive of the points about which a teacher would be concerned, 
but this Inclusiveness was not evaluated specifically for the five schools 
In this project. 

In the form used as part of the evaluation of this pilot training 
project, this questionnaire consisted of 30 Items. It Included Inquiry 
as to how helpful students had found the research course to be, how much 
Interest they had In research after taking the course, their reaction to 
teaching techniques, methods and content, and their own estimates of 
their knowledge gained. A number of questions Inquired specifically 
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about which parts of the content was most difficult or easiest, best and 
least liked, least and most useful in practice, taught best or worst, 
and requiring most and least study time. 

To increase the probable validity of responses, students were 
asked to complete this form anonymously. To permit some comparison of 
student course evaluations with grades, students who had received their 
final course grade were asked to put it on the course evaluation form. 

Teachers at the five schools, on their own initiative, had 
students complete this form in different ways. At school 5, students 
Completed the course evaluation the last day of the course when they 
had already received their course grades; at school 2, student grades 
were not ready and students completed the course evaluation the last 
day of the course without knowing their grades. At school 4, the 
evaluation was completed at the same time as the "after" MARK. At 
schools 1 and 3, the evaluation was completed in the Fall of 1966 when 
students had returned from the summer. 

Course Grades: 

Although grades earned in courses may be of dubious reliability 
and validity as a measure of knowledge acquired, they are a readily 
available and customary method of assessing students’ learning. 
Therefore, course grades were used in this project as one method to 
determine learning. 

A th^noretical 10 point scale was used to convert letter grades 
to numerical ones with A+ - 9, A - 8, A- - 7, etc., down to C- ■ 1 and 
D, E, or F ■ 0. No school made use of the full ten point scale, thus 
introducing an extraneous variable into the analysis of results. Two 
! schools' grades ranged over eight scale points, one school used five 
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tcalt pointty while two schools included only three and two scale points, 
respectively, grading only A, B+ and B, or A and B* 

Methods of Obtaining Data on Teaching Variables 
Data on teaching variables were obtained by content analysis of 
the sources listed below. 

School Catalogs: 

School catalogs provided infomiation on three variables, including 
the hours of credit for research, the period in the school year during 
which research was taught, and some information about the content of the 

course • 

Written Material from Teachers; 

Written material prepared by the 11 teachers whose classes partici- 
pated in this study covered two variables. These vere course objectives 
and the relative emphasis on various aspects of course content. 

Correspondence and Interviews With Assistant Project Directors. 

Material obtained by correspondence and interviews with the 
Assistant Project Directors at each school included data on ten variables. 
These were; any experimental objectives, content or methods; the evalua- 
tion and examination method used at the school; the learning experiences 
provided students; the background of the teacher including the level of 
the teacher’s training and the teacher’s field of specialization; the 
method of sectioning the course; and whether special classes were avail- 
able for above average students or remedial classes available for those 
with difficulty. 
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CHAPTER IV 



STUDENT CHARACTERISTICS PRIOR TO TRAINING 

This chapter presents information on student characteristics 
prior to taking the research course. A comparison of this sample with 
students in the Pins study of students in social work schools in 1960 
is made first. A profile of the typical student is then presented, 
followed by sections in the chapter on each of the variables on which 
data were obtained prior to training. In the interest of brevity, the 
l.D. material and data on interests presented in this chapter deal with 
all schools as a whole. Identifying data and interest variables found 
to predict differences in potential for learning research are compared 
school by school in Chapter VII. 

Because the MARK is the major predictor in this project, "before*' 
MARK scores are provided in more detail than the l.D. and interest data, 
with school to school comparisons being made. 

Comparison of This Sample With Students in 
Other Schools of Social Work 

Six identifying variables on which information was obtained for 
this project were categorized identically with that obtained in the 
Pins Study of 1960.^ 

^Arnulf M. Pins, "Who Chooses Social Work, When and Why," 

Council of Social Work Education, 1963. 
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While Infometlon on these six veriebles was obtained because of 
Its possible value for Identifying potential for learning research, the 
similarity of these variables to the Pins Study provided an opportunity 
to determine how representative this project sample was of social work 
students throughout the country. Such a comparison, of course, assumes 
there has been no change In student characteristics from i960, the date 
of the Pins Study, to 1965. the time at which these data were collected. 
The six variables coinnon to the training project and the Pins 
Study were parents’ Income, father’s occupation, father’s education, 
mother’s education, size of home town and the type of college or 
university from which student received an undergraduate degree^ 

This project sample scored somewhat above the Pins sample In 
respect to father’s occupation, and the education of both father and 
mother. Significant differences at the 5 per cent level were found. 
Tests by chi square showed no significant difference between the dls- 
trlbutlomln the Pins Study and this project on parent’s Income, size of 
home town, and type of undergraduate college. 

2 

Comparison of this group and the group In the Pins Study by X 
on grade point average, showed a significant difference at the 5 per 

cent level. The Pins group had a slightly higher undergraduate grade 

, 2 
point average. 

Typical Student Profile 

The father of the typical student was employed In a professional 
or managerial occupation, earned less than $7,000 per year, and neither 

^Because grades were not obtained In Identical categories In the 
two studies, this comparison necessitated some combining of categories 
In both studies. 
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he nor the mother was likely to have attended college. 

The size of the students* "home town" was most apt to be a city 
above 250,000 in population. Cultural resources of "home towns" were 
generally said to be "far below average." In contrast to this charac- 
terization, most students reported their own cultural background was 
somewhat broader than average. 

Education at a public university having a student body larger 
than 2,500, located in the southern United States was most character- 
istic of the group involved in this pilot training plan. The typical 
student was about as likely to have had a course in research, statis- 
tics, logic or scientific method as not to have had such a course. If 
he had taken any of these subjects, this was more likely because they 
were required than as an elective. 

As a rule, the typical student had avoided contact with mathe- 
matics and abstract reasoning in the past. Of those who had some 
experience with mathematics, about an equal number reported it had 
been pleasant as said it was unpleasant. 

About one-half of the students had been exposed to some class- 
room instruction, not always an entire course, related to research, 
usually in statistics. 

About as many student interests involved contact with people as 
did not. Many of the more frequently reported interests were oriented 
to cultural goals and could be considered to be more characteristic of 
women than men, such as listening to music, going to plays, reading 
poetry, etc. 

On the "before" MARK test, the typical student tended to give 
correct responses to about one-half of the knowledge items and to offer 
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responses considered desirable to about the satne proportion of attitude 
items. The modal total score was 29; the modal attitude and knowledge 
acores were 12 and 17, respectively. That the knowledge and attitude 
items appeared to measure different aspects of the student is evidenced 
by the rather low correlation between them, .14. 



I.D. Variables 



Parents' Income, Occupation, and Education: 

Almost one-half of the students came from homes where family Income 
was less than $7,000. (The average Income In the United Statea In 1965 
was about $7,300.) Exactly one-half of the fathers were either profes- 
sionals, proprietors, or In managerial occupations. Over two- thirds of 
the students' parents, both fathers and mothers, had no training beyond 
high school. 
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Cultural Background: 

Students tended to come from urban backgrounds. About one-half 
were from cities of 250,000 or above. 

Although slightly more than one-half of the students considered 
the adequacy of cultural resources In their home town below average, 
almost three-fourths of the students considered their personal cultural 
background broader than average. Where they received this additional 
cultural background Is not clear. 

In answer to a question about the amount of professional litera- 
ture In their homes, such os medical magazines, or business and engineer- 



ing journals, three-quarters of the students reported at least an 

l! 

occasional periodical or book of this kind* Only about one-half of the | 

I 

studpnre. hou*\i»r. renorted readina this material as much as occasionally* 
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Educational Background: 

Slightly less than half of the students, four out of ten, had 
gone to a public university and about one-half had attended a large 
college of over 2,300 students. The largest proportion of students, or 
about four out of ten, had gone to a southern college or university 
(perhaps reflecting the fact that there were two southern schools in 
this project). The next largest proportion had gone to an undergraduate 
college or university in the northern United States. These three ques- 
tions, as might be expected, produced most variation from school to 
school. 
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I About one-half of the students had received credit for an under- si 

1 

( 

graduate course with content covering in whole or in part statistics, j 

! 

research, logic, scientific method, or psychological tests. Two or 1 

j 

more hours of undergraduate credit in statistics had been received by | 

J 

more than half of the students. About two-thirds of those who took 1 

courses in research, statistics, or scientific method, had done so | 

because it was required. About half of those who took courses in logic j 

and tests and measurements did so as an elective. Three-quarters of 
the students had this kind of training within five years of attending 
a school of social work and auout half of them had taken these courses | 

within two years prior to entering graduate school. | 

j 

£ 

Background in Mathematics and Abstract Reasoning: \ 

I About half of the student group reported they usually avoided 

contact with math but approximately a third said they were neutral to, 
and about one-fifth reported a liking for this field. Students' atti- 
tude to math does not seem connected with their past experience, at 



least as reported in this study. Only one in five reported unpleasant 
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past experiences with math and one In six reported pleasant experiences, 
although about half had both pleasant and unpleasant experiences. The 
proportion who said they considered themselves to have about average 
ability In math (about half of the students) was about the same as those 

who liked It or were neutral to It. 

About as many students characterized themselves as preferring 
abstract reasoning as avoiding It. About six out of ten, however, said 
they preferred doing to thinking, and only 5 per cent preferred thinking 
to doing. The others were about equally divided on this question. 



Spare Time Activities and Interests 



The most comnon Interests of the students In this project 


did not 


•ear different from Interests of other young people their age. 
The ten spare time activities and Interests reported most 


often 


63 per cent or more of 
Listening to music 


the students were as follows: 
937. Sightseeing 


737. 


Reading news Items 


877. 


Recreation with children 


727. 


Reading non>flctlon 


837. 


Going to plays 


717. 


Bull sessions 


807. 


Swimming 


717. 


Travel 


777. 


Attending church regularly 


63% [ 


Reported by A3 per 


cent 


or more of the student group were 


these : | 


Walking 


6A7. 


Reading poetry 


32% 1 


Cooking 


637. 


Hiking 


327. j 


Visiting art museums 


377. 


Tennis 


A87. 


Reading history 


3A7. 


Planning community work 


A77. 1 


Singing 


337. 


Church social groups 


A6% 1 
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Activities and interests checked by from 10 to 20 per cent of the 
group were as follows: 



Sailing 


197. 


Political campaigning for 
a candidate 


177. 


Painting 


197. 


Auto mechanics 


14;L 


Hospital volunteer work 


187. 


Ceramics 


137. 


Studying the stock market 


187. 


Woodworking 


127. 


Folk dancing 


177. 


Betting on horse races 


117. 


Activities and interests 


least typical of the student group 


and 


checked by less than 10 per cent of 


the sample were: 




Boy or girl scout leader- 


107. 


Chemistry 


97. 


ship 




Physics 


57. 


Shell collecting 


107. 


Philately 


57. 


Flower arrangement 


107. 


Gem collecting 


27. 



Background in Research 

With regard to students' background in research as measured by 
the "before" MARK, considerably more similarity than difference was 
found between each of the five schools. This similarity appears 
sufficiently close to say that students at the schools were matched so 
far as attitudes to research and knowledge of research are concerned. 

This similarity in background in research is shown by the fre- 
quency distribution on "before" MARK scores in Tables 1 and 2 and the 
lack of significant difference between schools on both mean and median 
scores. 

While there is a significant difference at the 5 per cent level 
beti^en the mean scores of the lowest schools, 1 and 2, and the highest 
school, 4, this is barely significant and it is not significant at the 
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one per cent level. Differences between other mean scores were not signifi- 
cant for any of the other schools with regard to total scores, attitude 
scores or knowledge scores. 



TABLE 1 

MEAN SCORES ON "BEFORE” MARK TEST BY SCHOOLS 



School 


Mean 

Total 


Stand. Dev. 


Mean 

Atti- 

tude 


Stand. Dev. 


Mean 

Know- 

ledge 


Stand. Dev. 


1 


28.1 


5.4 


12.4 


2.7 


15.7 


4.5 


2 


28.1 


4.5 


12.4 


2.2 


15.8 


4.1 


3 


30.5 


6.4 


12.8 


2.2 


17.7 


5.5 


4 


30.8 


3.5 


12.6 


2.4 


18.3 


2.9 


5 


28.2 


5.1 


12.8 


2.4 


15.4 


4.0 








TABLE 2 










MEDIAN SCORES ON "BEFORE" MARK TEST BY SCHOOLS 


School 


Median Total 


Median Attitude 


Median Knowledge 


] 


L 


28.0 




12.0 


16. 


,0 




5 


28.0 




13.0 


15. 


,0 




J 


29.0 




13.0 


18. 


,5 


4 


30.0 




13.0 


19. 


.0 


C 




29.0 




12.0 


16. 


,0 



.f 
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CHAPTER V 



THE TEACHING VARIABLES IN THE TRAINING PLAN 



Differences snd Siwilflrities in Objectives » 
Methods and Content 



This chapter describes objectives sought at each school in the 
beginning research class» during this project^ and the content and 
methods used to reach these objectives. It includes a statement of any 
experimentation by the schools. Because the selection of schools 
attempted to achieve maximum variation in school size, geographic loca- 
tion covered, source of support and length of accreditation, similari- 
ties among the five schools should be considered a finding rather than 

an outcome of the selection process. 

Whenever possible, objectives and the relative emphasis on each 
within a school were obtained directly from written statements currently 
in use at respective schools. Early in the training plan, in September 
1966 , Assistant Directors from all schools, except school 2 , met at 
Tulane and discussed the relative emphasis at their respective schools 
on the various objectives listed below. These discussions permitted 
a consensus on which schools had least and most emphasis on each 



objective. 

Relative emphasis on course content was determined from state- 
ments in school catalogs, and by comparing these with class outlines and 
schedules when these were available. Inferences from this data were 
checked with Assistant Directors at the meeting given above, until 
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consensus wss obtsined on the relstive en^hasis given to various aspects 
of content at each school. School 2 later provided in writing informa- 
tion on objectives and content. 

Relative emphasis on teaching methods, methods of evaluating 
learning experiences, and teacher characteristics were obtained from 
Assistant Directors. The lack of specific detail on objectives and con- 
tent precluded analysis of possible relationships between emphasis on 
teaching specific content and change scores for certain MARK items that 
could be expected to reflect this teaching. Nevertheless, the use of 
global and general divisions of objectives and content was considered to 
maximize the reliability of decisions about relative emphasis on these 
from school to school. 

A detailed written statement of objectives and content for each of 
the three sections was not submitted by school A. Conclusions about 
school 4 are derived from general information obtained through the 
Assistant Director for that school. 

All of the information obtained from the five schools shows con- 
siderable agreement on the objectives of the research course and almost 
as much agreement about its content. There is considerable difference, 
however, in the emphasis given various aspects of content, and in the 
teaching methods used and learning experiences provided.^ Schools are 



^One of the difficulties in collating and comparing material con- 
cerning course objectives is that content of written statements on these 
topics makes use of a high degree of abstract terminology. Another 
problem is the tendency to state objectives in terms of learning experi- 
ences. For exaiq>le, the writer was not sure whether the objective 
"understanding the application of the problem-solving methods to profes- 
sional problems" is the same as "understanding of research as a problem- 
solving method" or "understanding the scientific method of problem- 
solving." Interviews with representatives of schools followed by some 
correspondence with individual teachers were used to reduce errors as 
much as possible in this step of the analyses. 
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most different from each other in methods of giving assignments and of 
evaluating learning, in teacher characteristics, and In the time and 
place of the research course in the curriculum. 

As far as could be determined from available information, there 
appears to be enough similarity in objectives and content so that the 
difference in other variables found may be looked at in terms of their 
influence on learning achieved in the course. 

Objectives: 

Courses at all of the schools share the goal of aiding the student 
to help clients better; the development of knowledge for understanding 
and interpreting research studies; and all seek to some extent at least, 
to prepare the student to carry out an actual research project or prac- 
ticum. 

A clear difference in emphasis on these objectives emerges from 
content analysis of schools' statements and from discussions with repre- 
sentatives of schools. 

School 5 places somewhat more stress than any other school on 
the course enabling the student to help clients. There is a further 
difference on the emphasis in understanding and interpreting research 
studies. Schools 2 and 3, and possibly 4, place more emphasis on this 
than schools 1 and 5. Schools 1,2,3, and 4, place considerably more 
emphasis on preparing a student to carry out a research project, than 
school 5. These differences are summarized in Table 3 that follows. 
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TABLE 3 

COURSE OBJECTIVES 





Schools with 
Most Enq>hasis 


Schools with 
Least Emphasis 


Preparing Students for 
Practice in Service 
Me thods 


3 


1.2, 3,4 


Developing Knowledge and 
Skills in Evaluating 
Research Studies 


2.3,A 


1.5 


Preparing Students to 
Carry Out a Research 
Project 


1 »2 


3 



Methods: ^ 

Schools with similar emphases on objectives were found to use 
Similar teaching methods, to offer similar learning experiences, and to 
evaluate learning alike; schools with different emphasis on objectives 
were different in regard to these other variables. Schools 1 and 3 were 
most likely to be together, or at least tssst likely were different from 
schools 2,3, and A. Schools 1 and 3 made most use of lecture and demon- 
stration by the teacher and gave more tests of an objective nature. 
Others used seminar or class discussion methods and written narrative 
type assignments more often. 

Tables A, 5, and 6 which sunnarize the comparisons among the five 
schools on these topics should not be interpreted as meaning that a 
school fails to engage in a particular teaching method, that it does not 
provide a particular kind of learning experience, or that it fails to 
use a particular type of evaluation. The tables also should not be 
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interpreted to indicate any relative emphasis wi thin a particular school 
on one or more of the points below« In addition^ the conclusions below 
are presented in this general form rather than in any more specificity 
as a means of maximizing their reliability. 



TABLE 4 

METHODS OF TEACHING 





Schools with 
Most Emphasis 


Schools with 
Least Emphasis 


Lecture 


1,5 


2,3,4 


Questions from class answered 
by class 


2 , 3 • and 
possibly 4 


1,5 


Questions from class answered 
by teacher 


1,5 


2,3, and 
possibly 4 


Demonstrations by teacher 


1,5 


2,3 


Exercises in class 


3,5 


1,2,4 


Exercises at home 


1,2, 3, 4 


5 


TABLE 5 






METHODS OF EVALUATING LEARNING 






Schools with 
Most Emphasis 


Schools with 
Least Emphasis 


Objective tests 


5,2 


1,3,4 


Papers (less scope than 
term papers) 


1,3,4 


2,5 
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TABLE 6 

LEARNING EXPERIENCES 





Schools with 
Most Emphasis 


Schools with 
Least Emphasis 


Listening to lectures 


1.2,5 


3,4 


Class discussion 


1.3,4 


2,5 


Reading research studies 


1.2,3 


4,5 


Writing or oral analysis 
of studies or reports 


2,3 


1.4,5 



Content: 

The emphasis on content, shown below, somewhat follows the emphasis 
on objectives and methods. 

TABLE 7 

CONTENT OF RESEARCH COURSES 





Schools with 
Most Emphasis 


Schools with 
Least Emphasis 


Statistics or Quantitative Methods 


1.2,3 


4.5 


Substantive Knowledge of Past and 
Current Social Work Research 


2,3 


1.4.5 


2 

Phases of the Scientific Method 


1.4.5 


2.3 


3 

Techniques and Instrumentation 


2.5 


1.3.4 



^Phases of the scientific method included such content on formu 
lating hypotheses, selection of data sources and observational units, 
methods of data collection and research design. 

techniques and instrumentation included such topics as how to 
make rating scales, how to test for reliability and validity, etc. 



A1 



Teacher Characteristics: 

Schools 1,4, and 5 had the nest experienced teachers. These 
teachers were in general also the teachers with the most advanced train- 
ing, and those whose training was in social work or in a field closest 
to social work. 

TABLE 8 

TEACHER CHARACTERISTICS 





Schools Where 
Teachers Had Most 


Schools Where 
Teachers Had Least 


Education and Training 


l.A,5 


2,3 


Teaching Experience 


1,4,5 


2.3 


Background like Social Work 


1.2,5 


3.4 


Full Time Work 


1,4,5 


2,3 



Miscellaneous Variables: 



Miscellaneous variables such as hours of credit, and how early in 
the students' educational experience the research course occurred, etc., 
were less patterned than the other teaching variables mentioned above. 

TABLE 9 

MISCELLANEOUS VARIABLES 





Schools With 
Most 


Schools With 
Leas t 


Hours of Credit 


2,5 


1.3,4 


Earliness of Occurrence of the 
Course in School Curriculum 


1.2 


3.4,5 


Size of Class 


1.5 


2,3,4 


Special Remedial Work 


1.2,5 


3.4 


Later Elective Hours of Work 
Avai lable 


2,3,5 


1.4 



/■ 
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Experimentation Introduced as Part of the 
Pilot Training Plan 

As part of the design of this pilot training plan, two schools, 2 
and 5, Introduced some experimentation. 

At school 2, students were divided Into four approximately equal 
sections. High scoring students on the "before" MARK were placed In 
Section A; low scoring students were put In Section B. Sections C and 
D each had a mixture of high and low scoring students. Statistics was 
added to the course content for students In Section A; students In 
Section B had no Instruction In statistics; and Sections C and D 

received the usual content and emphasis, which Included some statistics. / 

At school 5, the top 20 per cent of the students, according to 
"before" MARK scores, were put In Section A. Other students were assigned 
at random to Sections B and C so that each had about 40 per cent of the 
students. 

In addition to the difference In size and students' test scores. 

Section A was different than other sections In two other ways: (1) 

There was considerably more discussion and Interaction among the students 
themselves and less lecturing by the teacher. (2) Additional content 
was Introduced that dealt with conceptualization, and specialized 
research techniques and measurements, content often considered more 

i Suited for advanced research courses. 

1 

I At schools 1,3, and 4 student asalgnment to various sections was 

I at random. 

i 

I 

I 







CHAPTER VI 



EVALUATION OF INSTRUMENTATION AND DATA COLLECTION 

Most of the discussion on the evaluation of instrunents in this 
chapter deals with the "before" and "after" MARK, since this instrument 
played dual roles of major importance in the study. With course grades 
as the criterion, the "before" MARK was used both to identify various 
levels of student potential and to predict various levels of learning 
achieved. The "after" MARK served as a measure cotmon to all schools to 
validate course grades, and also, when compared with the "before" MARK, 
to show differences in learning that had been achieved. 

Evaluation of the MARK includes studies of its reliability, 
validity, homogeneity, difficulty, and ability to discriminate. The 
construction of the other instruments, with the exception of the 
special test questions, did not permit quantitative evaluations of 
reliability. Comments on the reliability of these instruments is thus 
based on logical analysis; tests of validity and discrimination v«re 
handled conventionally. 



The MARK 

During the six years the MARK was under development, its content 
was changed several times to increase its power to predict course grades 
at Tulane. Changes were not necessarily aimed at making it more repre- 
sentative of content taught either at Tulane or at other schools. 
Analysis of the content of the MARK used in this project, however, 
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according to the categories used in Chapter V to describe the content of 
the beginning research course as taught at the various schools of social 
%rark, indicates a fairly balanced sampling of the four content areas 

described. 

Ten iteiim on the MARK required knowledge of statistics or quanti- 
tative nethods; six items related to knowledge of findings of past and 
current research; twenty-one items asked for knowledge of various phases 
of the scientific method; and seven items referred to techniques and 
inatrumentation. The remaining fifteen items were all attitude items 
that did not appear to fall into any of these categories. 

This relatively greater emphasis on the general aspects of the 
scientific method and the relatively lesser emphasis on substantive 
knowledge of findings from research studies, and techniques and methods, 
represents the relative content emphasis in the beginning research 
course at Tulane. Though this distribution of the content measured by 
the MARK did not necessarily fit so closely the content emphasis at 
the other four schools, it did not significantly effect results. When 
the Tulane students were removed from the sample, for example, and 
analysis made of the other four schools, findings at the other four 
schools remained consistent with those from Tulane. 

The ten items about statistics or quantitative methods included 
seven questions requiring knowledge of terminology such as -parameter,*' 
•'mean,*' -correlation," etc., and three questions on statistical proce- 
dures such as how to make a frequency distribution or the function of 
statistics in research. 

The questions on knowledge of past and current research findings 
asked for general information about the findings of social work research 
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rather than specific knowledge obtained by any particular study. These 
questions covered such topics as whether social work research appeared 
to provide definitive or limited answers to problems in the field, how 
helpful social work research findings appeared to be in guiding practice, 
or how one evaluated the findings of a research study, etc. 

The twenty-one questions on the phases of the scientific method 
included general inquiries about the method as a whole and specific 
questions about certain aspects. Examples of general questions were 
items that asked why the scientific method was considered self corrective, 
and whether research in social work was different from that in other 
fields. Specific questions about phases of the scientific method included 
inquiries into what was the first decision to be made in a research 
study, or what criteria topics for a research study must meet. 

Questions on techniques and instrumentations were specific "how 
to do it" items. Examples of these were items asking how one differ- 
entiated abstract and concrete concepts, what guides were used in 
deciding how many classifications should be made from a given number 
of observations, how the validity of observations was determined, etc. 

i 

I 

In other work on the MARK, the writer has found that test items 
which appeared conceptually to cover similar content did not predict 
equally well. Because this finding leads to some question whether 
items which appear conceptually similar are actually measuring the same 
thing, no MARK subscores according to the above breakdown of items by 
content were developed. In further work on the MARK, factor analysis 
will be carried out. Subscores will then be computed for items that 
are found to measure common factors and the relation of these sub- 
scores to variation in content at the five schools will be studied. 
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Reliability— "Before" and "After" MARK; 

The reliability of the "before" MARK of .62 Is considered to show 
a satisfactory consistency In measuring potential for learning research. 
The reliability of the "after" MARK as a measure of what was learned In 
the research course is even more consistent, .75. The reliability of 
the "before" MARK was somewhat less than the reliability of Its parts; 
the reliability of the "after" MARK on the other hand, was somewhat higher 
than the reliability of the after attitude scores, but somewhat lower than 
the reliability of the after knowledge scores. These findings are shown 
In Table 10. 

TABLE 10* 

RELIABILITY OF "BEFORE" AND "AFTER" MARK SCORES 



Test 


Reliability 


Test 


Reliability 


TOTAL BEFORE 


.62 


TOTAL AFTER 


.75 


Attitude Before 


.66 


Attitude After 


.57 


Knowledge Before 


.76 


Knowledge After 


.84 


N - 263 








*A11 reliabilities 


are based on 


Pearsonlan correlations 


carried out 



using the split-half method of correlating scores on even numbered 
Items with scores on odd numbered Items. All are corrected for the 
number of Items, using the Spearman-Brown formula, so that reliabili- 
ties of attitude and knowledge Items are those that would be found If 
there were 39 attitude and 39 knowledge Items. 



The Increased reliability of the total scores and knowledge scores 
after students had taken the research course Is what would be expected on 
the grounds that there would be less guessing on the various Items after 
students had studied research. The reason why the reliability of atti- 
tude scores decreased after taking the research course Is unknown. One 
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Course in research may upset some previously formulated student attitudes 
about research but not necessarily help students to formulate others to 
take their place. Thus, at the end of one research course, student 
attitudes may be somewhat in a state of flux. If this is true, then 
studies of attitudes after the practicum that follows the research 
classroom course might show more consistency. 

As might be expected, school to school variation in the reliabili- 
ty of the MARK occurred to some extent. These could be largely accounted 
for by the size of the sample at the different schools. Most variation 
from school to school occurred in the reliability of before attitude 
scores and least variation in after knowledge scores. 

Item Difficulty and Heterogeneity "Before'' MARK: 

Some indirect evidence of the potential of the "before" MARK to 
identify various levels of potential for learning research can also be 
found in the relative difficulty of items and in the heterogeneity of 
the content.^ The distribution of items by difficulty, as shown in 
Table ll below, indicates the high variation in the difficulty of both 
attitude and knowledge items. 



^Tests with items of varied difficulty and heterogeneous content 
have been found to be most successful in identifying various levels of 
potential for learning the content in other subject areas. See Lee J. 
Cronbach, Essentials of Psychological Testing . New York: Harper and 

Brothers, I960. 






TABLE 11 



LEVEL OF 


ITEM DIFFICULTY—" 


BEFORE" MARK 




Level of Difficulty as 
Shown by Per cent of 


Number 


of Items at Difficulty 
Level Shown 


Students Who Missed the 


Total 


Knowledge 


Attitude 


Item 


Test 


Items 


Items 


80 - 997. 


10 


7 


3 


60 - 797. 


12 


9 


3 


40 - 597. 


15 


8 


7 


20 - 397. 


13 


8 


5 


0 - 197. 


9 


4 


5 


Total 


59 


36 


23 



The heterogeneity of a test may be measured by the number of Inter- 
correlated Items. Items having little or no Inter-correlatlons may be 
assumed to measure different things. 

The heterogeneity of the "before” MARK Is shown by the fact that 
less than 6 per cent of the Inter-ltem correlations are significantly 
greater than zero at the 5 per cent level. At the 1 per cent level, less 
than 1 per cent of the total Inter-ltem correlations are significant. 

The size of Inter-ltem correlation also Indicates relatively 
heterogeneous content. The highest Inter-ltem correlation was only .42, 
and out of a total of 710 possible Inter-ltem correlations, only four 
were above .30. 

t 

Discrimination: 

The possibility of the "before" MARK Identifying those students 
with different potentials for learning research Is a function of Its 
capacity to discriminate among students. 

1 
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Over 80 per cent of 48 out of the 59 items on the "before" MARK 
discriminated among the students in some way.^ As one might expect with 
a test given the students before they took the course, most items dis- 
criminated the top third from the other groups of students and fewest 
discriminated the bottom third from the other two-thirds, as shown by 
Table 12 below. 



TABLE 12 

ITEM DISCRIMINATION --"BEFORE" MARK 







Discriminated Top 
Third From Middle 
Third and Middle 
Third from Lower 
Third 


Discriminated 
Top Third 
From Other 
Two-Thirds 


Discriminated 
Lower Third 
From Other 
Two-Thirds 


Number of 


Items 


16 


24 


8 


Number of 
Items 


Attitude 


4 


12 


3 


Number of 
Items 


Knowl edge 


12 


12 


5 



Validity: 

Support for the validity of using the MARK as a measure of learn- 
ing achieved in the research course is found in the relatively high 
correlation (.56) of the "after" MARK with the usual measure of research 
learning, course grades, and from the discrimination of its items, their 
heterogeneity and level of difficulty# 



2to detemlne the dlscrlmlnetlen of MARK IteeB, the students «re 
vided into three groups according to total scores, °P ’ 

dJle tiJS, and tSe lower third. The discrimination of items shown 
i Table 12 indicates those items that discriminated the three groups 
L; ..eh Lher and those Items that discriminated one group from the 



Other two groups. 
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Since obtained validities are Influenced by the reliability of 
both of the variables that are correlated, In Table 13, below, both the 
obtained validity, and the validity corrected for the lack of reliability 
of the lnter>correlated variables are shown* 

TABLE 13 

VALIDITY OF "AFTER" MARK SCORES AS SHOWN BY CORRELATION 
WITH RESEARCH COURSE GRADES 







Obtained 

Validity 


Corrected for 
Attenuation 


TOTAL AFTER 




.42 


.56 


Attitude After 


, - 


.27 


.47 


Knowledge After 




.41 


.49 



N - 263 



In making the correction for attenuation (column two above). It 
w&s assumed that the course grades had the same reliability as the MARK. 
Because many course grades are based on term papers or narrative examl* 
nations. It seems likely that most course grades have a lower reliability 
than the MARK. The correction made Is thus on the conservative side and 
may tend to underestimate the true validity of ths !'^RK. At the school 
that used wholly objective tests to evaluate learning achieved, for 
example, the correlation of "after" MARK total scores with course grades 

after correction for attenuation was .77. 

School to school variations in the validity of the MARK as a 
whole were found also, largely related to the size of the sample at 
different schools and whether the grading was based on objective tests. 



short answer examinations, or on papers. 
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The relatively lower validity of the attitude items tray be due to 
the fact that they are actually of lower validity. This may also be due 
to the fact that the attitudes which the writer considered were likely 
to be associated with success in research were not necessarily those 
which were so associated. Further study of attitude items under 
another scoring system will be discussed in the forthcoming material on 
the MARK. 



Item Difficulty and Heterogeneity- -''After” MARK: 

There was almost as high a variation in difficulty of the "after” 

mark items as for the before form. Changes as might be expected are 
toward a lower level of difficulty as shown in Table lA. 

TABLE lA 

LEVEL OF ITEM DIFFICULTY— ”AFTER”MARK 



Level of Difficulty as 
Shown by Per cent of 
Students Who Missed the 
Item 


Number 


of Items at Difficulty 
Level Shown 


Total 

Test 


Knowledge 

Items 


Attitude 

Items 


80 - 997. 


10 


3 


7 


0 

1 


A 


A 


0 


AO .. 597. 


16 


12 


A 


20 - 397. 


19 


11 


8 


0 - 197. 


10 


6 


A 


Total 


59 


36 


23 



The heterogeneity of the "after” MARK items to social work students 
is, as might be expected, less than the "before” version. Sixteen per 
cent of the items are signif icsi-itly inter-correlated at the 5 per cent 
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level. At the one per cent level, ten per cent of the average Intar- 
Iten correlations are significant. 

The average Inter-ltetn correlation on the "after" MARK was also 
Considerably higher than on the "before" form of the test. 

The number of discriminating Items on the "after" NARK was almost 
the same as on the "before" MARK, 47 or 80 per cent. The number which 
discriminated among all three thirds, however, was considerably larger 
and the separation between the top two-thirds and the bottom ihlrd Is 
much more marked as shown In Table IS. 

TABLE 15 



ITEM DISCRIMINATION— "AFTER" MARK 



I 






Uiscrimina<:ea Top 
Third From Middle 
Third and Middle 
Third From Lower 
Third 


Discriminated 
Top Third 
From Other 
Two Thirds 


Discriminated 
Lower Third 
From Other 
Two Thirds 


Number of Items 




26 


4 


17 


Number of Attitude 
Items 




6 


2 


9 


Number of Knowledge 
Items 




20 


2 


8 



The I.D. Questionnaire and Interest Test 
The reliability of the I.D. Questionnaire and Interest Test must 
rest on methodological grounds. The fact that these have been used for 
several years by the %n:lter would tevtd to support the probability that 

particularly ambiguous Items or those likely to be answered Inconsistently 

\\ 

\^uld have been eliminated. The validity of these two Instruments was 
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unknown at the time data was obtained. Approximacely 20 per cent of the 
I.D, items correj^ted significantly with ’’after" MARK scores or course 
grades. Approximately the same proportion of interest items correlated 
with these variables. 

Some indirect evidence of how well these two instruments discrimi« 
nated among students might be deduced from the degree of correlation with 
another discriminating Instrument like the "before" MARK. Because these 
two instruments, however, were aimed at measuring different variables 
than the "before" MARK, correlations with the MARK were smaller in number 
than would otherwise be the case. Stronger evidence for discrimination 
was thus shown by the fact that students were relatively well distributed 
on each of the questions in both of these questionnaires. 

Forty per cent of the 32 items on the I.D. Questionnaire cot re • 
lated with the "before" MARK; ten out of 6A interest items or 16 per 
cent correlated with this instrument. 

The Special Test Questions 

The special test questions were found to have a spl^«hal£ 

reliability of .6A, corrected for the number of items. Their validity, 

as Measured by their correlation with more established measures of 

learning, like the research course grades was .3A. Their validity as 

3 

measured by their correlation with the "after" MARK was .33. The 
discrimination and item difficulty of the special test questions were 
similar to that of the "before" MARK test, with item <Ufficulty being 
Somewhat greater. 



^These correlations are corrected for attenuation of the cri- 
terion. Course grades were assumed to have reliability equal to the 
special test questions. 
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The Course Evaluation 

The reliability and validity of the course evaluation, like the 
I.D, Questionnaire and the interest test, depends on its methodological 
Soundness. Its use for several years eliminated a number of vague and 
dubious questions. Though correlation with other variables was pre> 
eluded because students were not identified, the students* feeling of 
anonymity would be expected to increase validity of responses. Findings 
in previous years that student responses to this instrument in writing 
agreed with their oral evaluation of the course and the teacher would 
provide further support for its reliability and validity. The relatively 
large range of responses indicates satisfactory discrimination. 

Research Course GradeiS 

Unlike other measures mentioned above, the research course grades 
were not obtained by a single instrument. Further, they ware computed 
by different teachers on the basis of varied information, and sometimes 
made use of what information was available with different degrees of 
precision. 

The combining of students from the five schools had the problem of 
Combining grades that w?re obtained on different bases and tended to move 
them toward the lowest common denominator of reliability and validity. 

This step, however, had at least the advantage of biasing the outcome in 
a conservative direction. Thus any prediction of grades from the "before" 
mark scores would mean that the MARK could be expected to predict some* 
what better if there were more reliable, valid, and discriminating criterion 
measures available of what had been learned. 
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Because some information on the reliability and validity of grades 
at the five schools was considered necessary to interpret the findings 
from this training plan, the writer made an attempt to categorize the five 
schools in this respect. In making these categorizations the writer 
assumed that grades based on objective tests were more likely to be valid 
than those based on short answer tests and that these in turn were 
probably more valid than those based on papers. It was further assumed 
that among schools using the same method of evaluating learning, those 
with the most experienced teachers, with teachers who had the most 
training in research, and whose field was closer to social work, would 
make the most valid evaluations of learning achieved. There is, of 
course, always the possibility that an experienced teacher can grade more 
reliably or validly on a term paper than an inexperienced teacher can 
construct and grade an objective test. On the basis of these assumptions, 
however, schools 2 and 5, and possibly school 1, are considered to have 
the most reliable and valid grades; schools 3 and 4 are considered to 
have grades which are least reliable and valid of the five. 















CHAPTER VII 

in’; OUTCOME and evaluation of the project 

This chapter discusses the extent to which the two major objectives 
of this pilot training plan were reached. It includes an assessment of 
how viell various levels of potential for learning research could be 
identified and what kind of teaching methods and content appear most 
likely to maximize each of the various potentials. It notes the rela- 
tion between the students' evaluation of the course and measures of 
learning achieved. 

Material showing the ability of the I.D. and interest data and the 
"before" MARK scores to identify and predict potentials for learning 
research is presented first. "After" MARK scores, course grades, and 
special test questions scores serve as criteria for these predictions. 
Analysis is then made of teaching methods to determine which best 
appears to maximize potential for all students, followed by analysis of 
those teaching methods that appear particularly useful for students at 
various levels of potential. In these latter two analyses, measures to 
evaluate the outcome of the pilot training plan are differences between 
the "before" and "after" MARK scores, special test questions scores, and 

student reports on course evaluations. 

Some contamination of the study's outcome was considered possible 
because the MARK was originally designed at Tulane to measure potential 
for learning among the writer's students and to determine how this 
potential was maximized by the content of the beginning research course 
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as conceptualized by the writer at Tulane. This meant that correlations 
between the "before" and "after" MARK, between the "before" MARK and 
course grades, and changes in MARK scores, for example, might be higher 
at Tulane than at other schools. To test whether this situation signifi- 
cantly influenced results, all tests made on the five schools were 
repeated on the other four schools alone, with the Tulane students ex- 
cluded from the sample. These tests did reveal the expected higher corre- 
lations and scores at Tulane. Nevertheless, all findings that were signifi 
cant with the Tulane students included in the sample remained significant 
at the other four schools, with the Tulane students excluded. Thus any 
bias on the findings from the fact that the test was originally designed 
for Tulane and the method of teaching used there can be considered to be 
minimal. In the interest of brevity, findings on the four schools other 
than Tulane are not presented separately from those for Tulane and all 
findings in this chapter refer to all five schools as a group, unless 
specific results are labeled differently. 

Identification of Levels of Student Potential 
For Learning Research 

The writer considers that this pilot training plan was successful 
in showing that various levels of potentials for learning research could 
be identified and learning achieved by students at these levels predicted. 
Some evidence comes from both I.D. and interest data but the major identifi 
cation comes from the "before" MARK scores. 

Identification and Prediction by I.D. and Interest Data: 

Though only a few of the I.D. and interest items identified levels 
of potential for learning research, the findings from those items which 
permit differential prediction are somewhat in line with findings from 
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previous studies. 

The l.D, items that predicted for all levels of students were those 
with face validity, and those found in these other studies to represent 
the students' social and cultural background and social cIass. 

Three of the interest items that predicted show a liking for risks, 

2 

billiards, fishing and playing poker. 

Table 16 below shows the correlations significant at the 5 per cent 
level between interest and l.D. items on the one hand, and the three 
criterion variables showing levels of learning achieved. Because these 
correlations are each based on only one item, they are considerably 
smaller than correlations between MARK scores and the same criteria for 
learning research. 



^Robert H. Knapp, op.cit . 

2oavid C. McClelland, "The Calculated Risk: An Aspect of Sclentifl 

Performance," Taylor and Barron, op.ci t. , pp. 184-217. 
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TABLE 16 

CORRELATIONS OF INTEREST AND I.D. ITEhB WITH COURSE GRADES, 
"AFTER” MARK SCORES, AND SPECIAL TEST QUESTIONS SCORES 



4 



Correlated With 



Item 



"After" 

MARK 

Scores 



Course 

Grades 



I.D. ITEMS 

Reaction to abstract reasoning .14 

Previous course In research .18 

Previous course In research as 

an elective *12 

Previous course In logic .14 

Previous course In logic as an 

elective *14 

Education of father ‘H 

Father's occupation 
Mother's occupation 

INTERESTS 



,11 



.11 

.11 



Special 

Test 

Questions 



Cheml stry 


- 


.12 


- 


Baseball 


• 15 


m 


• 


Coin collection 


.11 


.17 


• 


Billiards 


.11 


.17 




Hiking 


- 


• 11 




Reading poetry 


.17 


- 


• 13 


Square dancing 


- 


• 17 


• 15 

t c 


Flower arranging 


- 


- 


• 15 


Fishing 


m 


• 16 


m 


Auto mechanics 


m 


• 12 


• 


Reading science fiction 


- 


• 18 


- 


Playing poker 


m 


. 18 


m 


Reading biographies 


- 


• 16 


• 16 


Reading science 


m 


• 17 


m 


Church singing 


- 


- 


• 11 



When Interest Items showing a liking for risks were combined to 
make a "risk score" this score correlated much more strongly with the 
criterion variables above than any of the Interest Items alone. This 
risk score was the number of Interests that the students reported that 
appeared to Involve risks. The maximum score was 6, shown by Interests 
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in "studying the stock market," "betting on horse races," “playing 
poker," "playing bridge," "mountain climbing," and "fishing." Though this 
risk score was correlated only .18 with the final course grade it was 
correlated .36 with the "after" MARK score. 

Identification of Potential Levels by "Before" MARK Scores: 

In terms of their "after" MARK scores, the "before" MARK appeared 
to divide students into three groups. Slightly more than half of those 
in the top third on the "after" MARK scored in the top third on the 
"before" MARK; almost half of the middle third on the "after" MARK were 
in the middle third on the "before" test; almost two-thirds in the lowest 
third on the "after" form of the MARK came from the lowest third on the 
"before" form. Details are in Table 17 below. School to school varia- 
tion in this pattern was small and not statistically significant. 

TABLE 17 

RELATION BETWEEN STUDENTS ' RANK ON "BEFORE" OR "AFTER" MARK 



Rank on "Before" MARK 


Rank 


on "After" 


MARK* 




Top 


Middle 


Lower 




Third 


Third 


Third 


Top Third 


517. 


407. 


97. 


Middle Third 


307. 


457. 


257. 


Lowest Third 


197. 


157. 


667. 



*This kind of division seemed to produce more homogeneous groups than a 
more conventional division into quartiles. 
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Prediction of Learning by "Before” MARK Scores: 

The potential of the "before” MARK to predict learning In research 
that was expected from the discrimination of the Items, was borne out by 
Its correlation with the three criteria for learning used In this project. 

Prediction of "after" from "before scores . The ability of the 
"before" MARK to Identify what kind of "after" MARK scores students will 
make Is somewhat better than Its ability to predict course grades, as 
shown by the correlation between "before" and "after" MARK scores In 
Tables 18 and 19 below. 

The undergraduate grade point average predicts "after" scores 
considerably less well than "before" MARK scores. 



TABLE 18 

PREDICTIVE POWER OF "BEFORE" MARK SCORES WITH THE 
"AFTER" MA&K SCORES AS A CRITERION* 



Before Scores 


Total Score 
After 


Attitude 
Score After 


Knowledge 
Score After 


Total Scores 


.69 


.65 


.57 


Attitude Scores 


.37 


.35 


.30 


Knowledge Scores 


.65 


.61 


.53 


Undergraduate GPA 


.28 


.03 


.37 



*A11 correlations have been corrected for attenuation of the criterion 
according to the reliabilities given In Chapter VI. 



Prediction of grades from "before" scores . Though the "before" 



MARK correlates less well with grades than with "after" scores, its 
predictive power with grades Is comparable to or better than most tests 
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In use In other fields, as shown In Table 19 below. In this case also 
the ’'before” MARK predicts significantly better, at the 5 per cent level, 
than the undergraduate grade point average. 

TABLE 19 

PREDICTIVE POWER OF "BEFORE” MARK SCORES WITH COURSE 
GRADES AS A CRITERION* 



Correlation with 
Course Grades 



Total Score *37 
Attitude Score *19 
Knowledge Score 

Undergraduate GPA *22 



♦These validities corrected for attenuation In the criterion only, repre- 
sent the predictive power of the "before” MARK scores If grades had a 
reliability equal to that of total "before" MARK scores (.62). No 
correction for the reliability of the MARK has been made. Grades 
probably have a lower reliability than .62 so this estimate Is on the 
conservative side and probably under-estimates the predictive power of 
the MARK. (For example, at the one school that used objective tests 
as the sole method of evaluating learning, the corrected correlations 
were .53, .32, and .47, respectively.) 



Prediction of Special test questions scores from "before” scores . 
The ability of the "before" MARK to predict those students who did best 
on the special test questions Is evidence of Its ability to Identify the 
group of students who might learn more than usually taught In the basic 
research course. These t«ould be the students who could be expected to 



^For example, the Miller Analogies, widely used In predicting 
success In Clinical Psychology, has been shown to have a correlation of 
.35 to .45 with course grades In psychology courses. 
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become researchers when they completed their Master's training. 

"Before" MARK scores again predict special test questions scores 
considerably better than the undergraduate grade point average, as shown 
In Table 20 below. 



TABLE 20 

PREDICTIVE POWER OF THE "BEFORE" MARK WITH SPECIAL 
TEST QUESTIONS AS A CRITERION* 





Corrected for 




Attenuation of 




Criterion 


Before Total 


.31 


Before Attitude 


.22 


Before Knowledge 


.21 


Undergraduate GPA 


.08 



*All correlations have been corrected according to reliabilities given 
In Chapter VI. 



The Relation of the Teaching Variables to 
Various Levels of Potential 
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This part of the analysis deals with the outcome of the teaching 
In this pilot training plan. It assumes that the MARK Is the best 
measure In this study that can be applied to all student^ and that the 
achievement of potential Is best measured by the change In MARK scores, 
particularly MARK knowledge scores, as explained below. The special 
test questions scores, on the other hand, are assumed to be the best 
measure of achievement of potential for the superior or outstanding 
group of students. 

The course evaluation provided a self-assessment by students of 
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the learning they thought they had achieved, measures of their interest 
In the course, and their opinion on the usefulness of the content. In 
addition. It provided sot* data on how students viewed the teachln^v 
methods, and the amount of time they spent In studying. 

This section, unlike the previous sections of this chapter, which 
have discussed the five schools as a whole, deals with differences among 
the five schools In the teaching variables and their relationship to 
these criterion measures. Because of. the difficulty In quantlfi.'lng data 
dealing with teaching methods, course content, and emphasis, this section 
has used dichotomous break-downs of the gains In learning mdo by 
students as compared to dichotomous break-downs on the teaching variables. 

Similarity of the '’Before" and "After" Sample; 

The group who took the "before" and "after" MARK are considered to 
be representative of typical students In the five schools of social work 
under study despite the loss of 90 students between the two test 
administrations. The rationale for this conclusion is given below. 

A test by chi square showed there was a significant difference 
between the group of 90 students who did not take the "after" MARK and the 
group of 263 students who took both the "before" and "after" MARK. This 
difference was significant at the 5 per cent level but not at the I per 
cent level. Comparison of the distributions of the two groups showed 
that almost all the difference In distribution was attributable to nine 
students In the group who did not take the "after" MARK. These nine 
students had scored extremely low on the "before" MARK also had very low 
undergraduate grade point averages and did not complete their graduate 
social work education. Thus, the group who took both the "before" and 
"after" MARK are considered representative of typical students at the five 


















schools Involved. 

Differences Between "Before" and "After" MARK Scores; 

Table 21 below shows that for four schools there was a signifi- 
cantly greater than chance (at the 5 per cent level or above) Increase 

^ 5 

In MARK scores after the student had taken the research course. 



TABUS 21 

INCREASE IN TOTAL MARK SCORES AFTER STUDENTS 
HAD TAKEN THE RESEARCH COURSE 



School 

Number 


Increase 
In Mean 
S core 


Increase 
In Median 
Score 


Level at Which 
Mean Increase 
was Significant 


Level at Which 
Median Increase 
was Significant 


1 


3.A 


3.0 


.05 


.05 


2 


3.1 


3.5 


.001 


.001 


3 


2.9 


A.5 


.05 


.01 


A 


0.9 


1.0 


- 


- 


5 


9.1 


9.0 


.001 


.001 


Almost all of the Increase In MARK scores was due to an Increase 


In knowledge 


scores; attitude scores at four of the five 


schools showed 


a decrease, 


although this change was significant at only two of the 


schools, as 


shown In Table 22 below. 


(Changes In median 


scores ««re 



Adhere were many reasons why the other 81 students did not take 
the "after" MARK. Some were 111 or absent at the time It was admlnls- 
tered.> Some did not wish to conq)lete It. 



^Soine of the lock of significant Increase In school A may be due 
to the fact that only a third of the original students completed the 
"before" and "after*’ MARK under conditions similar to that at other 
schools. \ 
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quite similar to changes In mean scores and In the interest of sli^llclty 
are omitted.) 



TABLE 22 



INCREASE IN MEAN ATTITUDE AND KNOWLEKE SCORES AFTER 
STUDENTS HAD TAKEN THE RESEARCH COURSE* 



School 


Increase In 


Level of 
Slanlf Icance 


Increase In 
Knowledge 


Level of 
Significance 


1 


-0.6 


.10 


3.0 


.001 


2 


-0.5 


.10 


3.5 


.001 


3 


-1.7 


.01 


4.6 


.001 


4 


-1.7 


.01 


2.6 


.10 


5 


l.O 


.01 


8.0 


.001 



*Mlnuses before numbers Indicate a decrease In mean scores. 



The decrease In attitude scores, or the failure of attitude 
scores to Increase, may be related to the lack of reliability of the 
attitude part of the MARK or to other factors. Some of these other 
factors may be the students' attempt to show attitudes they thought 
were desired, or the fact that some schools stressed different attitudes 

than the ones used In scoring the MARK. 

Frequency distributions for each of the five schools showed 
similar changes In range of scores. To preclude Identification of 
schools by sample slse, frequency distributions are not listed here. 

The "before" MARK total scores ranged from 12 to 41, out of a 
possible score of 59. Twenty-two scores were below 20 and only three 
were 40 or above. Out of a possible score of 59 on the "after" MARK, the 
range was 16 to 51; only five were below 20, and 30 students scored 40 
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or above. 

Relation of Increase In MARK score to students * rank on "before** 
MARK . A disturbing Inverse relationship was noted between the amount 
learned, as shown by differences between "before" and "after" MARK know* 
ledge scores and the rank of students on the "before" MARK. If the goal 
of the research course Is to maximize learning for the top third as well 
as for the other two thirds of the students, rather than bringing each 
student up to some minimum standard, this situation presents a particular 
problem for the profession. 

Students In the top third have the lowest gain In knowledge, as 

t 

shown by MARK scores; students In the bottom third have the most gain; 
while students In the mld*thlrd were between these two groups. See 
Table 23 below. 






TABLE 23 

INCREASE IN MARK KNOWLEDGE SCORES BY STUDENTS' RANK 



Students' Rank 


Mean Knowledge 
Score "Before" 
MARK 


Mean Knowledge 
Score "After" 
MARK 


Increase 


Top third of students 


20.1 


23.3 


3.2 


Middle third of students 


15.9 


21.0 


5.1 


Lowest third of students 


11.6 


19.0 


6.4 



When comparisons between sections are made at the two schools that 
assigned students to sections by "before" MARK scores, schools 2 and 5, 
the same Inverse relationship between rank on the "before" MARK and 
learning achieved Is found, even though one school showed a considerably 
higher gain than the other. See Table 24. 
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TABLE 24 

INCREASE IN MARK SCORES— STUDENTS SECTIONED BY RANK 



Sections 


Mean "Before" 
MARK 


Mean "After" 
MARK 


Increase 


High sectlon-School 2 


17.8 


19.6 


1.8 


Low sectlon-School 2 


14.3 


17.1 


2.8 


High sectlon-School 5 


18.9 


24.9 


6.0 


Low sectlon-School 5 


11.4 


21.0 


9.6 



The failure of students who scored higher on the "before" MARK to 
gain as much as their colleagues who scored lower on the "before" MARK 
does not appear to be due to the Inability of the MARK to measure learning 
achieved. Even those students who scored highest on the "after" MARK* 
missed from 10 to ^5 of the 59 questions so If they had additional know- 
ledge, there was opportunity for them to Indicate this by answering these 
questions correctly. 

Instead, the relatively small gain of those students scoring high 
on the "before" MARK appears partly related to the level of content In 
the research course. 

The writer believes that the relatively high anxiety students 
express about learning research and the fear of some faculty for them In 
the course both tend to reduce the level at which the research course Is 
taught. This climate, plus the boredom at hearing already known material, 
may well result In only a little learning among students with potential 
for much more gain. 
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Relationship Between Changes in MARK Scores 
and Teaching Variables at the Five Schools: 

Changes in median knowledge scores v«re used as the major variable 
in analyzing the outcome of this pilot training plan. Median scores were 
selected because they were less influenced by unusually outstanding or 
limited student?.. Mention is made, however, of findings that are supported 
or not supported by changes in mean scores. 

Conclusions suffer from all of the problems of this project design 
which tried to exploit individual differences among schools and sections 
as experimental variables. The writer recognizes that conclusions are made 
in considerable ignorance of interaction effects among the various vari- 
ables, and of unknown influences from uncontrolled variables. Neverthe- 
less, they appear to offer a beginning point for further pilot training 
plans or research studies. 

Median knowledge scores increased most at schools 2 and 5 and 
least at schools 1,3, and 4. Mean scores increased most at schools 3 
and 5 and least at schools 1,2, and 4. Thus school 5 had the greatest 
increase in both mean and median scores, while schools 1 and 4 had the 
least increase in these two scores. 

The material below first presents findings for each school as 
compared to each other school. These are followed by comparisons of 
sections within schools. 

The method of teaching seemed to make only slight difference in 
student learning, but schools with most didactic teaching made slightly 
higher scores. The most significant finding about teaching methods was 
that exercises at home appeared to have little influence on knowledge 
as measured by the MARK. 
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Schools who used objective tests had more Increase In knowledge 
scores than those who used papers as a bases for grades. 

Listening to lectures appeared to be the best leanxlng experience 
for obtaining an Increase In knowledge scores. 

Emphasis on content that covers techniques and Instrumentation 
appeared to make for most learning as measured by the MARK. 

Teacher characteristics made very little difference In MARK 
scores. Most Increases in scores were at schools with teachers who had 
moderate experience. 

More hours of credit and o»re hours of additional electives open 
to a student In research tended to be associated with Increased student 
learning; courses early In the students' educational experience were 
associated with less changes In MARK scores than those which occurred 
later. The often controversial factor of class size appeared to make 
very little difference In gains In knowledge as measured by the MARK; 

If there was any difference^ It slightly favored the larger classes. 

Details on these findings follow: 

Teaching Methods . With the exception of exercises In class, 
didactic teaching appeared to have a slight advantage over discussion 
methods of presenting material. Exercises In class appeared to have 
considerably more association with Increase In MARK scores than exer- 
cises at home. Least Influence on changes In MARK scores came from 
class demonstrations by the teacher. These findings were supported by 
changes in both median and mean scores. 

Learning Experiences . Findings on learning experiences fitted In 
with the findings about teaching methods. More changes In median and 
mean MARK scores occurred at those schools where students' learning 
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exptrlencet «ere largely listening to lectures In comparison to those 
which had more discussion. The relatively minor pertinence of resiisa 
research studies as a learning experience to changes In MARK scores may 
be due to the failure of the MARK to Identify this kind of learning; the 
analysis of studies on the other hand, appeared to be associated with 
upward changes In both mean and median scores. Schools that emphasized 
analysis of studies had more change In these scores than those which did 

not. 

Evaluating Learning Achieved . Those schools with the most objective 
measures of evaluating learning made most gain In median MARK scores, thos 
with least objective measures of evaluating learning made least gains In 
nedlan scores. Changes In mean scores were less consistently associated 

with these findings, but followed this pattern. 

Content . All schools that emphasized research techniques and 
Instrumentation had larger Increases In median scores than schools which 
do not. This finding was upheld to a partial extent with regard to mean 
scores. Least pertinent content to Increasing MARK scores was statistics 
and phases In the scientific method; schools that emphasized these most 
had less gain In both median and mean scores than schools which emphasized 
them least. Content highlighting the findings of research studies was 
only slightly associated with changes In MARK scores. 

Teacher Characteristics . Gains In MARK scores were only slightly 
related to teacher characteristics. Less knowledge as measured by the 
MARK was gained In schools with teachers who had most and least training 
and experience than where teachers had moderate training and experience. 

Miscellaneous Variables . Four of the miscellaneous variables had 
relationships to changes In MARK scores. Consistently, In those schools 
which had the most hours of credit, gain In MARK scores was highest. 
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Where the number of hours of credit for research courses was smallest, 
the change In MARK scores was least. At schools providing most remed- 
ial work and the greatest number of elective hours of research on com- 
pletion of the basic course, there was a greater gain In median scores 
than In schools offering less. On the other hand, where research was 
taught early In the students' educational experience, there was less 
gain In MARK scores than In the schools where It was taught later. 

Relationship Between Changes In MARK Scores and 
Teaching Variables In the Fourteen Sections: 

Except when differences were deliberately Introduced for experi- 
mental purposes, all participating schools made considerable effort to 
control differences In course content, emphasis, and requirements for 
papers and examinations. In order to keep Inter-sectlon differences with- 
in each school to a minimum. 

Earlier mention has been made that the relatively small Increase 
In knowledge by the top third of students on the "before" MARK (as 
measuri'd by differences between "before" and "after" knowledge scores) 

appeared to be a function of the content of the conventional research 

6 

course. 

What the additional content should be for students with some back- 
ground In research is not yet clear, but It appears likely that these 
students can and will learn more about how to conceptualize, the 
relation between theoretical and empirical concepts, and techniques and 
Instrumentation In research than they will about statistics. 

^Thls finding Is similar to that of L.D. Jaffe, In "Building an 
Undergraduate Social Work Research Sequence," Journal of Jewish Communal 
Services, Vol. 42, No. 1, 1965, pp. 99-108. | 
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At school 2, for example. Section A had considerable statistical 
content added as an additional requirement. Despite the fact that about 
20 per cent of the MARK measures knowledge of statistics, this section 
gained less In knowledge as shown by MARK scores than any other section 
at any other school. At school 5, Section A had considerable additional 
content added about theoretical and empirical conceptualization and 
research techniques and Instrumentation that was considered to be at an 
advanced level. These students made a greater gain on the MARK In both 
mean and median scores than any other section except Section C at school 
5. Some of the students In Section C had considerable remedial help from 
tutors and this assistance may have been responsible for this relatively 
greater Increase In scores. 

Influences of teacher differences are most noticeable when 
sections are examined. Sections that had teachers with social work back- 
ground generally gained more than sections with teachers from some other 
field. Within a school, sections that had more lecture appeared to gain 
more than those with more discussion. 

Relation Between Change In MARK Scores and 
Special Test Questions: 

The lack of relationship between special test questions scores 
and changes In MARK scores would Indicate that the STQ*s measure some- 
thing other than that measured by the MARK. Further support for this 
belief is shown by the findings that those students who scored highest 
on the "before" MARK (and thus showed most potential for learning 
research) had higher scores on the STQ*s than those students who scored 
lowest on the "before" MARK. 

Special test questions scores were obtained at only four schools. 
School 4 did not give this test. 
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There was little relation between the median change In MARK scores 
and the STQ scores. School 5 was high on both the change In MARK scores 
and the STQ scores; school 1 was low on both scores. School 3, however, 
was low on the change In MARK scores and high on the STQ scores, school 
2 was high on the change In MARK scores and low on the STQ scores. 

Students who scored In the highest third on the "before” MARK made 
special test questions scores significantly higher (at the 5 per cent 
level) than those who scored In the lowest third on the "before" liARK. 

The gain In students' scores on the "after" MARK was negatively associated 
with their STQ scores. That Is, those students who made the most gain 
In "after" MARK scores had the lowest STQ scores or those who made the 
least gain had the highest STQ scores. 



Relation of Special Test Questions Scores and 
Teaching Variables by Schools: 



Only a few of the teaching variables were related to the special 
test questions scores. 

The schools with most exercises In class (3 and 5) and with 
teachers with more social work background (1 and 5) ranked higher on 
the special test questions than those with least class exercises or 
teachers from outside social work. Schools with fe%iest lectures and 
those with most emphasis on reading research studies and statistics, 

(2 and 4) made lower special test questions scores than the others. 

Section to section variations In special test questions scores 
were similar to those among schools. Section A at school 5 In which 
additional advanced level content was added, scored highest on the 
STQ's. Students In Section A at school 2 where content on statistics 
was added, scored lowest of all sections on STQ's. 
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Students' Responses to the Course Evaluation 
Students* Assessment of the Knowledge They Gained: 

After completing their research course work, only three out of ten 
students believed they were ready for an advanced course In research; one- 
half were uncertain and the remainder did not feel ready. About half of 
the students felt ready to participate In research as a member of a team, 
and three out of ten thought they could carry on a research project with 
some consultation. 

Usefulness of Students' Learning to Their Practice: 

Almost eight out of ten students considered the research course 
less useful to practice than other courses, but six out of ten reported 
they received more help from this course than they had expected. Six 
out of ten students believed that research should be a required rather 
than an elective course. Nine out of ten students were of the opinion 
that research should come In the first year of the Master's program and 
should be of at least a full semester's duration. 

Students' Interest In the Course: 

Responses on three Items measuring Interests were almost Identi- 
cal to those on Items measuring usefulness of learning to practice. 

Students' Reaction to Teaching Methods: 

Only about one-half of the students felt as free to raise ques- 
tions In class as they would have liked, but seven out of ten found 
teachers* answers to questions as helpful as could be expected. With 
regard to specific teaching methods such as lectures and discussions, 
students were generally satisfied with what they were getting. Half of 
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the students » however, wanted more demonetratlone by the teacher and 
exercises In class and fewer exerclaea at home. About the same propor- 
tion Indicated a preference for a seminar or class smaller In size than 
theirs. Class slue was reported as the chief obstacle to student learn- 
ing. As might be expected, this reaction was much more prevalent In 
schools with larger classes. 

Seven out of ten students reported they spent less than three 
hours In preparation for class. The average preparation for examina- 
tions was an additional eight to twelve hours of study. 



Students' Characterization of Course Content: 

Students' ratings of the course content with regard to Its use- 
fulness to their practice, difficulty In learning, liking by the student 
for the content, material requiring most study time, and the quality of 
5;he teaching showed much variation.^ 

, Content about parameter-statistic relationships was characterized 
most frequently as most difficult, least liked, taught least well, and 
requiring the most time for study. No other content was characterized 
with any similar outstanding frequency. Least difficult content, 
requiring fewest hours of study was sampling. Most useful In practice 
was reading and analyzing research reports. Teachers were said to 



^Topics from which students could make a choice In these charac- 
teristics Included the following: scientific approach to knowledge, the 

cycle of scientific Inquiry, question and hypothesis formulation, 
standards for observations, methods for making observations, connecting 
questions and hypotheses to observations, qualitative classifications, 
rating scales, sampling, research design, empirical conceptualization, 
nwasures of central tendency and variation, relationships between 
variables, parameter statistic relationships (significance tests), 
theoretical conceptualization, connecting theories and observations, 
schedule making, research Interviewing, writing research proposals 
and research consultation, reading and analyzing research reports. 
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teach question and hypothesis fonoulatlon best. No single content area was 
an outstanding favorite. Writing research proposals and materials on 
research consultation was liked slightly more than other topics. 

While It Is possible that some students at schools other than the 
writer's Interpreted the meaning of categories of content given them 
differently than students at the writer's school » school to school compari- 



sons show considerable similarity, as shown by Table 25 below. 

TABLE 25 

SCHOOL TO SCHOOL COMPARISONS ON STUDENT CHARACTERIZATION 

OF COURSE CONTENT 



Most dlff lcult--parameter statistic relatlonshlps--Schools 2,3,4, and 5; 
scientific approach to knowledge —School 1. 

Least dlff icult— measures of central tendency— Schools 1,2, and 3; 
sampling. Schools 2,4; the cycle of scientific Inquiry, School 5. 

Liked best— no consensus--sclentlf Ic approach to knowledge. School 1; 
writing research proposals at School 2; reading research reports at 
School 3; no consensus at School 4; scientific approach to knowledge 
at School 5. 

Liked least-- parameter statistic relatlonshlps--all schools. 

Most useful — reading and analyzing research reports — Schools 1,2,4, and 
5; connecting theories and observations. School 3. 

Least useful — parameter statistic relationships— all schools. 

Most time spent studylng--no consensus-- scientific approach to knowledge. 
School 1; measures of central tendency. School 2; research design. School 
3; reading reports. School 4; research Interviewing, School 5. 

Least time studylng--no one topic received a majority vote at any school. 

Taught best— no consensus — scientific Inquiry, School 1; design. School 2; 
question and hypothesis formulation. School 3; cycle of scientific 
Inquiry, School 4; writing research proposals. School 5. 

Taught least well — parameter statistic relationships. Schools 2,4, and 5; 
scientific approach to knowledge. School 1; empirical conceptualization. 
School 3. 
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Relationship Between Change in MARK Scores 
and Students' Evaluation of Course: 

There was almost no relationship between change in MARK scores by 
Schools and the way schools were divided on the items used in the 
students' evaluation of the course. For two items where a relationship 
was present) one was negative and the other positive. There was a 
slight negative relationship bet:^en the students' report of interest in 
research) for example) and the change in MARK scoreS) with less interested 
students showing the most change. On the other hand) there was a slight 
positive relationship between the changes in MARK scores and students' 
assessments of the usefulness of the course. 

Because course evaluations did not identify students by sections) 
no analysis of the relationship between findings from the course evalua- 
tion and sections was possible. 

Relationship Between Course Grades and Course Evaluations: 

Since each student recorded his research course grade on the 
evaluation) when it was available) it was possible to classify course 
evaluations with regard to whether they were by superior) average) and 
below average students. For this analysis) students with "A+)" "A" 
and "A-” (387.) were put in one group; students with "B-^" and "B" (A77.) 
in another; and the "B-)" "C" and "F" students (157.) made up a third 
group. In the material below) these groups are referred to as "highest)" 
"middle" and "lo^-jest" groups. Grouping students in this way by grades 
was considered to minimize problems from the reliability and validity 
of grades. 

Student Asses sment of the Knowledge They Gained . The lowest group 
had most conflict about their ability. They felt considerably less ready 
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to go on to an advanced course than the other two groups, but expressed 
the same degree of confidence as the other groups that they were ready 
to participate in a research project. 

Usefulness to Practice and Interest in Research . A larger pro- 
portion of the middle and highest group believed research should be 
required, but only the highest group wanted two semesters of research 
required instead of one. The middle group found the course more 
interesting than the other groups; a larger proportion of the highest 
group, than in the other tw, found it more interesting than they had 
anticipated. 

Reaction to Teaching Methods . The highest group" felt most free- 
dom to participate, but still felt most restricted by class size. They 
wanted less lecturing, fewer direct answers by the teachers to questions 
from students, more class discussion and smaller classes. They pre- 
ferred objective tests. 

The middle group found the teachers' answers to questions most 
helpful, wanted more demonstrations in class by the teacher, smaller 
classes and a slower pace to the course. They preferred that grades 
be based on both papers and tests. 

The lo\«est group felt most restricted by the pace of the course 
and wanted more exercises at home. 

Though students in the lowest group reported that they spent 
m»st time in preparation for class, there was little difference between 
the middle and highest groups on this variable. Time spent in preparing 
for examinations was inversely related to the rank of the three groups. 

Students* Characterization of Course Content . The three groups 
differed considerably in how they characterized content as shown in 
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Table 26 below. 



TABLE 26 

characterization of course content by student level 
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Highest 


Middle 


Lowest 


Most difficult 


Parameter- 

statistic 

relationships 


Parameter- 

statistic 

relationships 


Parameter- 

statistic 

relationships 


Least difficult 


Measures of 

central 

tendency 


Sampling 


Sampling 


Liked best 


Scientific 
approach to 
knowledge 


Methods for 

making 

observations 


Methods for 

making 

observations 


Liked least 


Parameter- 

statistic 

relationships 


Parameter- 

statistic 

relationships 


Parameter- 

statistic 

relationships 


Most useful 


Connecting 
theories and 
obse rvations 


Reading and 
analyzing 
research 
reports 


Reading and 
analyzing 
research 
reports 


Least useful 


Parameter- 

statistic 

relationships 


Parameter- 

statistic 

relationships 


Paramete r- 

statistic 

relationships 


Most time in 
studying 


Paramete r- 
statistic 
relationships 
and research 
designs 


Connecting 
theories and 
observations 
and reading 
reports 


Scientif i c 
approach to 
knowledge 


Least time in 
studying 


Sampling 
schedule making 


Research 
interviewing 
rating scales 


Sampling 
the cycle of 
scientific 
inquiry 


Taught best 


Cycle of scien- 
tific inquiry 


Design and ' 

cion formu- 
lation 


Scientific 
approach to 
knowledge 


Taught least 
well 


Parameter- 

statistic 

relationships 


Parameter- 

statistic 

relationships 


Connecting 
theories and 
observations 
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Relationship of Students' Evaluation of 
Learning and Teaching Variables by Schools; 

Student Assessment of_Knowledge_jrhe^f_Ga ine^ Didactic teaching 
appears to give students less confidence but a more realistic assessment 
of their ability and better preparation for other work. At schools with 
didactic teaching, students were more likely to report they were ready 
for an advanced research course than those with discussion teaching 
methods. Yet, students taught didactically, felt confident only to 
engage in research as a member of a team and not to do a complete study 
even with consultative help. The method of evaluating students' learning 
had no relation to students' assessment of their knowledge. Content about 
the scientific method seemed most likely to increase students' confidence 
in their knowledge and the reality of this confidence. 

The teacher characteristics of long experience, most training, 
full-time work, and social work background appeared associated with most 
realistic student assessment of their own research knowledge. Of the 
miscellaneous variables, only class size seemed associated with students' 
assessment of their knowledge gained, with larger classes having the 
most realistic assessment. 

Usefulness to Practice and Interest in Research . Schools with 
most discussion methods of teaching, course content from studies, less 
experienced teachers, and small classes reported most usefulness for 

content and most interest in it. 

Reaction to Teaching Methods . Students found teachers' answers 
more helpful in the class with most didactic teaching, where content 
emphasized phases in the scientific method, and where the teachers had 



more experience. 
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Students who were taught didactically spent more time in preparing 
for their courses than students in discussion classes. The same was true 
of students who were given objective examinations versus those who were 
asked to write papers. The only kind of content related to time spent 
in preparation was that relating to scientific method; students at 
schools with more emphasis on scientific method spent more time in 
preparation than at schools emphasizing other content. Students consis- 
tently spent more time in preparation for classes taught by teachers who 
were more experienced and had more training than with teachers of less 
experience and training. The only miscellaneous variables associated 
with time in preparation were class size and remedial work. Students in 
larger classes as well as those who had most remedial work appeared to 
spend most time in preparation. 

Students* Characterization of Course Content . Only one relation- 
ship bet^^een student characterization of course content and teaching 
variables at the various schools was found. While in all schools 
parameter-statistics relationships was the most difficult Irregardless 
of emphasis, students at those schools which emphasized statistics and 
quantitative measures found measures of central tendency and variability 
least difficult while others did not. 
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CHAPTER VIII 



DISCUSSION OF FINDINGS AND SUGGESTIONS FOR THE FUTURE 

The evaluation of this pilot training plan indicates that identi- 
fication of various levels of potential for learning research among 
social irork students can be made* This identification has somewhat 
more reliability and validity than the determination of those teaching 
variables that maximize learning for students at each of the various 
levels. Suggestions for current teaching, as well as for future pilot 
training plans, come from both findings. 

This project has provided evidence of the ability of the MARK 
test to divide students into at least three levels in terms of their 
potential for learning social work research. At the same time, it 
indicated that the MARK might discriminate better if some attitude 
items were changed or eliminated, and if more knowledge items were 
added. Further work on this instrument to implement these suggestions 
is planned and the companion monograph to be completed approximately 
July, 1968, should show how this improved form of the MARK can separate 
students more discriminately according to their potential for learning 
research. The second monograph will also attempt to explain why the 
MARK predicts and to develop some beginning theory on what makes for 
success in various aspects of social work research. 

Three Kinds of Student Potential for Learning Research 
The writer has noted three rather distinct kinds of students who 
appear to correspond generally to the three levels of potential for 
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learning research that have been Identified by the MARK. 

In the first groups are those students who scored highest on the 
MARK, scored somewhat better than average on the attitude Items, and 
whose performance In the research class Indicated a strong Interest In 
the "doing" aspects of research. 

In the second group, were those who scored almost as high on the 
MARK and whose potential for learning research was almost as good as the 
first group, but whose Interest In actually engaging In research activi- 
ties was considerably less. Although this second group had less Interest 
In actually engaging in research themselves, they recognized the Importance 



had more than a passive Interest In supporting research when It was 
carried out by other members of the profession. 

The third group had considerably less potential for learning 
research and almost no Interest In doing It. This group, however, showed 
some Interest In using the products of research, and a potential for 
learning enough research to use Its products effectively In their pro- 
fessional practice. 

The writer has designated these three groups as "doers," 
"supporters," and "users" on the basis of their Interests and potentials. 



Members of the "doer" group are seen as having a potential, with 
some consultative help, for engaging In social work research after com- 
pleting a Master of Social Work degree. 

The "supporter" group are those who could be expected to support 
and forward research activities carried out by others, although they will 
probably not engage In It themselves. The students In this group appear 
to be those who will quickly find their way Into supervisory and 



of research based knowledge to forward professional goals, and they thus 
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administrative positions after graduation. In these latter positions, 
they will be In a position to support and forward research carried out 
by others. 

The "user” group represents the third level of the research poten- 
tial Identified by the MARK. These are the students with limited ability 
to learn research but who appear able to use Its findings In their prac- 
tice. Use of research findings Is considered an Inter-actlonal process, 
so these persons are seen as more than "consumers" of research. They 
would be expected to Indicate how research findings met their practice 
needs and where findings needed reevaluation or extension. From this 
group one could expect suggestions that would guide the work of the 
"doers." 

Among these three kinds of students, the most Important finding of 
this study is the apparent limitations of current teaching methods and/or 
content to maximize the learning for the "doer" group. Evidence for 
this was the relatively small change between "before" and "after" MARK 
scores for the "doer" group, as compared with changes In these scores 
found In the "supporter" and "user" groups, and In the evaluative 
comments of all three groups. 

Students In the "doer" group not only gained less knowledge than 

^ I 

either of the other two groups but wanted more research courses. 

Students In the "doer" group were dissatisfied with the pace of the 










^hls finding is supported by a follow-up study of 83 students 
carried on at Tulane In 1966-67, data for which became available during 
the writing of this report. These 83 students were placed In sections 
according to their "before" MARK scores. The mean Increase In MARK 
scores In these four sections corresponding to the four quartlles on 
the "before" MARK was as follows: 1st quartlle 3.8; 2nd quartlle 8.1; 

3rd quartlle 11.2; and Ath quartlle 13.0. 
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couratt and Mntcd a faatar pact and anra content. They had leaa Intereat 
In the courae and found It leaa uaeful than other groupa. 

The finding that the high acorlng atudenta advanced leaat aupporta 
the writer 'a prevloua undocuaonted auppoaltlon that the potential of thoae 
atudenta who could learn moat waa not being realized. Concern about thla 
point waa one of the motivating factora for thla project. 

Thla Inverae relatlonahlp between the amount of knowlec''^^ held at 
the beginning of the courae and the gain In knowledge during the courae 
appeara to be aaaoclated with lack of aufflclent courae content to mee'e 
theae atudenta* needa; or If the content la In the course, thla Icaaer 
learning appeara related to lack of atlanilatlon of student learners. 

In 1962 through 1964, at Tulane, the writer tried teaching research 
to the atudenta with highest MARK scores In sections where they were 
mixed with other students; In 1965 and 1966 these high scoring students 
were placed In a section by themselves, and additional content was Intro* 
duced Into this section. As measured by change In MARK scores, the learn* 
Ing of theae high scoring students was leas than other students In both 
Instances. 

The writer now believes that some different approach Is necessary. 
He suggests these high scoring students on the MARK have the alternative 
of taking something other than a beginning course In research, either an 
Intermediate research course If this fits their Interests or some elective 
other than research If they are less Interested In research than some 
other area. 

Comments of those students who scored high on the "before" MARK 
Indicate that at least some of the failure of top*scorlng students to 
learn more la related to their lack of stimulation. Thla situation 
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occurr6d 6ven when addition®! material was added for thene students by 
the writer in 1965 and 1966, because some of them insisted that the 
course also include a review of beginning content. If high scoring 
students were permitted to have a choice of electing an intermediary 
course of research or some other kind of course, a more homogeneous 
group for each kind of course could be an outcome, with more stisiulation 
possible, and consequently greater learning. 

The special test questions scores show that high scoring students 
who can be considered better prepared can learn additional content even 
if it is part of a beginning course. If an intermediary course was 
built on what they already knew, was limited to students who elected it, 
and thus exploited their interests, greater learning could be expected. 
On the assumption that these students know what kind of teaching is 
best for them, it is suggested that any such additional course be 
taught in a discussion manner with objective examinations used to 
assess learning. 

Further study is needed of the influence of specific content 
emphasis on student learning. In this project not enough detail was 
available on specific content taught at the various schools, and not 
enough was known about which items on the MARK measured various content 
aspects to study this relationship. At the same time further experi- 
mentation can be carried out with special courses and content planned 
for the three groups of “doers," “supporters," and “users." 

A Suggested Curriculum to Maximize Student Potential 
Such a suggested experimental curriculum is outlined below. 
Further study can be made, both of the influence of specific current 
content emphasis, and this kind of specialized experimental teaching on 















student learning. The MARK end other Instruments developed In this study 
can be used as nessurlng devices of whet knowledge students gain. 

The writer visualizes s research sequence that will provide two 
semesters of three hours each of classroom work In research end with 
t«fo separate projects or experiences of at least one semester each. The 
first semester of classroom work would be required for all students except 
those In the top third on the MARK. These would be exempted, and for them 
the second semester would be elective. The first project would be required 
for all students; the second project would be elective. 

The first semester would have as Its objective the preparation of 
all three kinds of students for the various professional activities, of 
doing, supporting, and using research. Most emphasis would be given In 
this beginning course to preparing the studenta to be "users.” This stress 
would be based on the assumption that all students would be expected to 
be "users" to some extent. Content relating to doing, supporting, and 
using research would be Included, but content relating to using would be 
emphasized most. Teaching methods most appropriate for each kind of 
content would be used for the particular content Injected. Thus, the 
entire semester would not be lecture, or discussion, or exercises, etc., 
but presentation and learning experiences would relate to each of these 
three kinds of content and objectives. 

Student Interest In research and Integration of research learning 
with the rest of the curriculum %x>uld be forwarded by having a practlcum 
or 4‘esearch experience beginning either at the same time or shortly after 
the course began. This research experience would be closely related to 
the students' field Instruction so that the topic of research would emerge 
from the students' field placement experience. Field Instructors and 
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research Instructors could jointly work on this project. Field Instruc- 
tors could use the research experience to show how helping clients can 
be facilitated by research based knowledge; research teachers can use 
It to apply sotne of the theoretical aspects of research covered In the 
research class. 

Integration of research Into the field experience could be 
expected to provide a number of benefits to students. In the field, 
the student sees situations about which he needs more knowledge or 
aspects of which he wants to change. Engaging In a research project 
related to these situations should help the student to recognize the 
Importance of having knowledge before decisions are made, or to see the 
usefulness of knowledge In affecting change. 

Students at the other two levels, supporters and doers, would 
have additional learning experiences available as electives. Both 
groups would have an opportunity to elect a second research classroom 
course. Content of this course would cover such topics as how to 
obtain funds for research (and would Include exercises In writing 
grant requests), budgeting and administrative activity related to 
research, the role of research In the administrative structure of the 
agency, the use of research consultation, etc. 

Doers could elect this course plus a second practlcum, or sub- 
stitute field Instruction In research In their second year for field 

• 2 

Instruction in one of the other practice methods. 

^Tulane has begun to develop such a sequence on an experimental 
basis. Informal evaluation of student reaction to such a plan Is quite 
positive. Further work on this plan Is now going on and Interested 
persons can obtain more details from the writer. 
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Suggestions for Future Training Plans 

With this proposed curriculum as a basis, the writer suggests 
further pilot training plans with course content, teaching methods, 
learning experiences, etc., based on what has been learned so far In 
this project. 

A similar number of schools \rould appear appropriate for such a 
training plan; the size of the sample for this project was large enough 
for accurate statistical calculations but small enough to be manageable. 
More preparation for the training plan, such as more prior discussion 
between and among teachers about specific content and earlier planning 
by schools for experimental teaching could Increase the validity of 
findings. The design could also Include better control of certain vari- 
ables so that section to section evaluation %K>uld not be confounded by 
undesirable combinations of experimental variables. For example, compari- 
son between didactic and discussion methods of teaching could be made, 
with the size of the class held constant, or with a design that per- 
mitted variation In both these variables. Using both objective examina- 
tion and papers for the same class would permit comparisons of these 
methods of evaluating learning with learning as measured by the MARK 
scores. 

There Is evidence from the analysis of the outcome of this train- 
ing plan that additional content concerning conceptualization, the 
relationship between theoretical and empirical models, techniques, and 
methods of Instrumentation, obtaining funding for research, and the 
use of research consultation can all be Introduced for all three levels. 
Because material on statistics seems to Inhibit learning In the course 
some other method of Introducing It should be attempted. Tulane has 
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tried to limit the research classroom course to discussions of the 
meaning of statistics and Introduce the computations of statistics Into 
the project at the time when a student feels the need for such computa- 
tions to test his hypotheses. This change, however, has not been in 
effect long enough to permit evaluation of outcomes. 

It Is likely In carrying out such a training plan, that some 
students with weak backgrounds In research and scientific method, and 
with poor undergraduate records will have even more difficulty In the 
course than they do now. In that event, remedial help should be pro- 
vided, preferably by tutors recruited from second year students or 
students In an advanced program. Values of this tutoring plan would be 
threefold: assistance for the beginning students In difficulties; 

teaching experience for the students providing the help; and for the 
teacher, freedom to aim Instruction at a loftier target than the 
••lowest cotnnon denominator^^ of the class* 

Evaluation of this proposed training plan would be carried out 
In a manner similar to that used In this study, with the modified 
Instruments that will shortly be available. It Is expected that 
further Item and content analysis will result In scoring patterns 
particularly characteristic of the three groups, the "doers," the 
"supporters," and "users." These additional scoring patterns will 
provide further criteria for evaluation of the outcome of teaching. 
Further changes In the course evaluation form can fit It to changes in 
teaching methods and content. 
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